
Lead Software Security Engineer – AI Benchmark Delivery & Team Leadership
Posted 2 days ago

Posted 2 days ago
This is a fully remote position, open to applicants in Brazil, +5 more countries.
• Oversee a team of trainers tasked with developing training and evaluation assignments for advanced AI coding models.
• Organize task deliveries by batch, distribute work among pod members, and monitor daily output against phase-specific objectives.
• Assess pod performance, pinpoint bottlenecks, resolve delivery challenges, and ensure tasks are on track.
• Conduct weekly client updates, convey delivery status, and escalate any risks related to tooling problems, rate limits, safety blocks, and resource limitations.
• Collaborate with clients to clarify requirements and synchronize task scope, volume, complexity, and difficulty adjustments.
• Perform final evaluations to confirm tasks adhere to quality, security, reproducibility, and assessment standards.
• Train and calibrate Pod Leads and trainers on CWE classification, verifier robustness, task difficulty, technical specifications, and acceptance criteria.
• Leverage throughput and quality data to anticipate staffing needs throughout project phases.
• Launch hiring and onboarding initiatives approximately three weeks prior to anticipated capacity needs.
• Investigate vulnerabilities, replicate security issues, create proof-of-vulnerability tests, and develop or review benchmark tasks during peak workload periods.
• Keep documentation updated regarding delivery plans, risks, quality findings, throughput metrics, staffing projections, and project decisions.
• A minimum of 6 years of professional experience in software engineering.
• Expert-level knowledge in at least one of the following: Python; JavaScript / TypeScript; Java; C / C++; C#; Go; Rust; Ruby; PHP; Kotlin; Swift.
• In-depth understanding of software security concepts, including CWE, CVE, and the OWASP Top 10.
• Ability to replicate software vulnerabilities and create proof-of-vulnerability tests.
• Practical experience with security tools, including sanitizers, fuzzers, and static-analysis tools like CodeQL or Semgrep.
• Strong skills in Git, Linux, Bash, and Docker.
• Excellent technical judgment in assessing verifier robustness, security patches, and AI-generated code.
• Experience in reviewing technical work and maintaining consistent quality standards among various contributors.
• Some background in mentoring engineers, trainers, or technical contributors.
• Outstanding technical writing, documentation, and communication abilities.
• Capacity to work autonomously in a fully remote, research-focused setting.
• Proven experience on this project or in similar software security datasets, benchmark development, or security evaluation tasks.
• Competitive salary and performance-based bonuses.
• Flexible work hours and remote work options.
• Opportunities for professional development and training.
• Comprehensive health benefits and wellness programs.
• Collaborative and innovative work environment.
NetImpact Strategies Inc.
Vannevar Labs
GuidePoint Security
Checkly
Get handpicked remote jobs straight to your inbox weekly.