Lead Software Security Engineer – AI Benchmark Delivery & Team Leadership

Posted 2 days ago

This is a fully remote position, open to applicants in Pakistan, +5 more countries.

📋 Description

• Oversee a group of trainers tasked with creating training and evaluation activities for advanced AI coding models.

• Strategically plan task deliveries in batches, allocate responsibilities among team members, and monitor daily output against specific phase targets.

• Assess team performance, pinpoint workflow bottlenecks, address delivery challenges, and guarantee timely progression of tasks through production phases.

• Conduct weekly reports for clients, communicate the status of deliveries, and proactively highlight risks related to tooling challenges, rate limits, safety restrictions, and resource limitations.

• Work closely with clients to clarify project requirements and align on the scope, volume, complexity, and adjustments in task difficulty.

• Perform final evaluations of completed tasks to ensure compliance with quality, security, reproducibility, and evaluation standards.

• Train and calibrate Pod Leads and trainers on CWE classification, verifier robustness, task complexity, technical specifications, and acceptance criteria.

• Leverage throughput and quality metrics to predict staffing requirements across different project phases.

• Initiate hiring and onboarding processes approximately three weeks prior to anticipated capacity needs.

• Investigate vulnerabilities, replicate security issues, develop proof-of-vulnerability tests, and create or review benchmark tasks during high workload periods.

• Keep documentation updated regarding delivery plans, risks, quality assessments, throughput metrics, staffing predictions, and project decisions.


⛳️ Requirements

• Minimum of 6 years of professional experience in software engineering.

• Mastery in at least one of the following programming languages: Python; JavaScript / TypeScript; Java; C / C++; C#; Go; Rust; Ruby; PHP; Kotlin; Swift.

• Deep understanding of software security principles, including CWE, CVE, and the OWASP Top 10.

• Capability to replicate software vulnerabilities and create proof-of-vulnerability tests.

• Practical experience with security tools such as sanitizers, fuzzers, and static-analysis tools like CodeQL or Semgrep.

• Strong skills in Git, Linux, Bash, and Docker.

• Sound technical judgment in evaluating verifier robustness, security updates, and AI-generated code.

• Experience in reviewing technical work and ensuring consistent quality standards among various contributors.

• Some background in mentoring engineers, trainers, or other technical contributors.

• Exceptional skills in technical writing, documentation, and communication.

• Ability to work independently in a fully remote, research-focused environment.

• Proven experience working on this project or in similar software security datasets, benchmark development, or security evaluation tasks.


🏝️ Benefits

• Flexible working hours to promote work-life balance.

• Opportunities for professional development and growth within the company.

• Collaborative and innovative work environment.

• Competitive salary and comprehensive benefits package.

People also viewed

SHI International Corp.2 days ago

Solutions Executive – Security

US flagTexas OnlyFull-timeCybersecurity / Security Engineer$110k – $175k/year
ApplyView job
Anduril Industries2 days ago

Vulnerability Security Engineer

US flagUnited States OnlyFull-timeCybersecurity / Security Engineer$114k – $196k/year
ApplyView job
Hex2 days ago

Product Security Engineer

US flagUnited States OnlyFull-timeCybersecurity / Security Engineer$198.8k – $310k/year
ApplyView job
Mercola2 days ago

Senior AI Security Engineer

US flagUnited States OnlyFull-timeCybersecurity / Security Engineer$130k – $150k/year
ApplyView job
Red Hat2 days ago

Sales Specialist – Top Secret Security Clearance, SCI, CI Polygraph

US flagMaryland, +1 more stateFull-timeCybersecurity / Security Engineer
ApplyView job
Teladoc Health2 days ago

Staff Zero Trust Security Engineer

US flagUnited States OnlyFull-timeCybersecurity / Security Engineer$165k – $185k/year
ApplyView job

Never miss a great job!

Get handpicked remote jobs straight to your inbox weekly.

Trusted by 7,400+ designers