Lead Software Security Engineer – AI Benchmark Delivery & Team Leadership

Posted 2 days ago

This is a fully remote position, open to applicants in Brazil, +5 more countries.

📋 Description

• Oversee a team of trainers tasked with developing training and evaluation assignments for advanced AI coding models.

• Organize task deliveries by batch, distribute work among pod members, and monitor daily output against phase-specific objectives.

• Assess pod performance, pinpoint bottlenecks, resolve delivery challenges, and ensure tasks are on track.

• Conduct weekly client updates, convey delivery status, and escalate any risks related to tooling problems, rate limits, safety blocks, and resource limitations.

• Collaborate with clients to clarify requirements and synchronize task scope, volume, complexity, and difficulty adjustments.

• Perform final evaluations to confirm tasks adhere to quality, security, reproducibility, and assessment standards.

• Train and calibrate Pod Leads and trainers on CWE classification, verifier robustness, task difficulty, technical specifications, and acceptance criteria.

• Leverage throughput and quality data to anticipate staffing needs throughout project phases.

• Launch hiring and onboarding initiatives approximately three weeks prior to anticipated capacity needs.

• Investigate vulnerabilities, replicate security issues, create proof-of-vulnerability tests, and develop or review benchmark tasks during peak workload periods.

• Keep documentation updated regarding delivery plans, risks, quality findings, throughput metrics, staffing projections, and project decisions.


⛳️ Requirements

• A minimum of 6 years of professional experience in software engineering.

• Expert-level knowledge in at least one of the following: Python; JavaScript / TypeScript; Java; C / C++; C#; Go; Rust; Ruby; PHP; Kotlin; Swift.

• In-depth understanding of software security concepts, including CWE, CVE, and the OWASP Top 10.

• Ability to replicate software vulnerabilities and create proof-of-vulnerability tests.

• Practical experience with security tools, including sanitizers, fuzzers, and static-analysis tools like CodeQL or Semgrep.

• Strong skills in Git, Linux, Bash, and Docker.

• Excellent technical judgment in assessing verifier robustness, security patches, and AI-generated code.

• Experience in reviewing technical work and maintaining consistent quality standards among various contributors.

• Some background in mentoring engineers, trainers, or technical contributors.

• Outstanding technical writing, documentation, and communication abilities.

• Capacity to work autonomously in a fully remote, research-focused setting.

• Proven experience on this project or in similar software security datasets, benchmark development, or security evaluation tasks.


🏝️ Benefits

• Competitive salary and performance-based bonuses.

• Flexible work hours and remote work options.

• Opportunities for professional development and training.

• Comprehensive health benefits and wellness programs.

• Collaborative and innovative work environment.

People also viewed

NetImpact Strategies Inc.10 hours ago

Information Systems Security Manager

US flagUnited States OnlyFull-timeCybersecurity / Security Engineer$90k – $125k/year
ApplyView job
Vannevar Labs10 hours ago

Information Systems Security Officer

US flagUnited States OnlyFull-timeCybersecurity / Security Engineer$120k – $160k/year
ApplyView job
GuidePoint Security10 hours ago

Security Engineer, Operational Technology

US flagUnited States OnlyFull-timeCybersecurity / Security Engineer
ApplyView job
Checkly11 hours ago

Senior Product Engineer, Incident Response

US flagUnited States OnlyFull-timeCybersecurity / Security Engineer€79k – €107k/year
ApplyView job
GuidePoint Security11 hours ago

Senior Security Engineer – Active Directory

US flagUnited States OnlyFull-timeCybersecurity / Security Engineer
ApplyView job
AbbVie11 hours ago

Cybersecurity Architect

US flagIllinois OnlyFull-timeCybersecurity / Security Engineer$124.5k – $236.5k/year
ApplyView job

Never miss a great job!

Get handpicked remote jobs straight to your inbox weekly.

Trusted by 7,400+ designers