
Senior Software Security Engineer, AI Benchmark Development
Posted 2 days ago

Posted 2 days ago
This is a fully remote position, open to applicants in Brazil, +5 more countries.
• Research and analyze real-world vulnerabilities derived from CVEs, security advisories, bug reports, and open-source repositories.
• Classify and categorize vulnerabilities using the most precise applicable CWE and align them with relevant OWASP Top 10 categories when suitable.
• Replicate security issues and create dependable proof-of-vulnerability tests that reveal the underlying weaknesses.
• Develop Harbor-format audit-and-patch tasks across various programming languages and software ecosystems.
• Compose comprehensive task instructions that describe security issues without disclosing vulnerable code, exploit specifics, or anticipated remediation.
• Establish reproducible Docker-based environments featuring pinned dependencies, clear setup guidelines, and reliable execution processes.
• Design and strengthen automated verifiers to deter gaming, test manipulation, hardcoded solutions, and overfitting.
• Evaluate and score AI-generated patches and technical reasoning using structured security and engineering rubrics.
• Recognize tasks that can be solved through superficial pattern matching or by replicating known upstream patches without a thorough understanding of the vulnerability.
• Document the context of vulnerabilities, reproduction steps, expected behavior, patch requirements, testing methodology, and review decisions.
• Over 5 years of professional experience in software engineering.
• Advanced proficiency in at least one of the following programming languages: Python; JavaScript / TypeScript; Java; C / C++; C#; Go; Rust; Ruby; PHP; Kotlin; Swift.
• Strong grasp of software security principles and prevalent vulnerability classes.
• Hands-on knowledge of CWE, CVE, and OWASP Top 10.
• Capability to independently reproduce vulnerabilities and create proof-of-vulnerability tests.
• Experience with security testing and analysis tools, including: Sanitizers; Fuzzers; Static-analysis tools like CodeQL or Semgrep.
• Solid proficiency in Git, Linux, Bash, and Docker.
• Ability to craft automated tests and reproducible technical evaluation environments.
• Exceptional technical writing abilities, with a talent for clearly articulating complex security issues and remediation needs.
• Capacity to work autonomously in a fully remote, research-focused setting.
• Flexible working hours and the option to work remotely.
• Opportunities for professional development and continued education.
• A collaborative and innovative work environment.
• Competitive salary and performance-based bonuses.
NetImpact Strategies Inc.
Vannevar Labs
GuidePoint Security
Checkly
Get handpicked remote jobs straight to your inbox weekly.