
Lead Software Security Engineer – AI Benchmark Delivery & Team Leadership
Posted 2 days ago

Posted 2 days ago
This is a fully remote position, open to applicants in Pakistan, +5 more countries.
• Oversee a group of trainers tasked with creating training and evaluation activities for advanced AI coding models.
• Strategically plan task deliveries in batches, allocate responsibilities among team members, and monitor daily output against specific phase targets.
• Assess team performance, pinpoint workflow bottlenecks, address delivery challenges, and guarantee timely progression of tasks through production phases.
• Conduct weekly reports for clients, communicate the status of deliveries, and proactively highlight risks related to tooling challenges, rate limits, safety restrictions, and resource limitations.
• Work closely with clients to clarify project requirements and align on the scope, volume, complexity, and adjustments in task difficulty.
• Perform final evaluations of completed tasks to ensure compliance with quality, security, reproducibility, and evaluation standards.
• Train and calibrate Pod Leads and trainers on CWE classification, verifier robustness, task complexity, technical specifications, and acceptance criteria.
• Leverage throughput and quality metrics to predict staffing requirements across different project phases.
• Initiate hiring and onboarding processes approximately three weeks prior to anticipated capacity needs.
• Investigate vulnerabilities, replicate security issues, develop proof-of-vulnerability tests, and create or review benchmark tasks during high workload periods.
• Keep documentation updated regarding delivery plans, risks, quality assessments, throughput metrics, staffing predictions, and project decisions.
• Minimum of 6 years of professional experience in software engineering.
• Mastery in at least one of the following programming languages: Python; JavaScript / TypeScript; Java; C / C++; C#; Go; Rust; Ruby; PHP; Kotlin; Swift.
• Deep understanding of software security principles, including CWE, CVE, and the OWASP Top 10.
• Capability to replicate software vulnerabilities and create proof-of-vulnerability tests.
• Practical experience with security tools such as sanitizers, fuzzers, and static-analysis tools like CodeQL or Semgrep.
• Strong skills in Git, Linux, Bash, and Docker.
• Sound technical judgment in evaluating verifier robustness, security updates, and AI-generated code.
• Experience in reviewing technical work and ensuring consistent quality standards among various contributors.
• Some background in mentoring engineers, trainers, or other technical contributors.
• Exceptional skills in technical writing, documentation, and communication.
• Ability to work independently in a fully remote, research-focused environment.
• Proven experience working on this project or in similar software security datasets, benchmark development, or security evaluation tasks.
• Flexible working hours to promote work-life balance.
• Opportunities for professional development and growth within the company.
• Collaborative and innovative work environment.
• Competitive salary and comprehensive benefits package.
SHI International Corp.
Anduril Industries
Hex
Mercola
Get handpicked remote jobs straight to your inbox weekly.