Software Engineer, Benchmarking

This is a fully remote position, open to applicants in United States.

📋 Description

• Execute benchmarks: Integrate AI benchmarks into our evaluation framework (mainly utilizing the Inspect library) to enhance the range of capabilities we monitor. Enhance our current benchmark suite to enable swift and efficient assessments of new model releases.

• Create new benchmarks: Engage in the formulation of entirely new benchmarks. You'll have the chance to propose and prototype your own concepts while also assisting with ongoing initiatives.

• Collaborate: Partner closely with researchers, analysts, and fellow engineers at Epoch AI to ensure that evaluation data and outputs are accurate, insightful, and seamlessly incorporated into our research products and publications.

⛳️ Requirements

• Strong engineering foundation: A solid software engineering background with over two years of professional experience in developing and maintaining intricate systems. You are expected to consistently produce high-quality, robust, and maintainable code and be adept at delving into existing codebases and infrastructure.

• Ideas and innovation: Candidates should be capable of generating their own concepts for new benchmarks, experiments, and innovative projects.

• Mission-oriented: You are driven by Epoch AI’s goal to deliver thorough, independent insights into significant trends in AI. You aim to provide public, reliable evaluations of AI capabilities on demanding benchmarks, enabling researchers, policymakers, and the broader public to make informed decisions regarding AI.

• Expertise in AI or experience in cybersecurity are valuable assets but are not mandatory.

🏝️ Benefits

• Fully remote work environment, featuring flexible hours and schedules for most positions.

• Competitive global benefits package, which includes an extensive health insurance program—covering supplemental benefits specific to local regulations as applicable—and life insurance along with a pension plan when relevant in your country.

• Generous paid time off (PTO), with no specific cap on PTO, including 30 days per year safeguarded, unlimited personal and sick leave, and up to 6 months (a mix of paid and unpaid) parental leave for permanent employees.

• A flexible and generous expense policy, allowing you to invest in equipment and a wide array of productivity tools or learning and development opportunities that you find beneficial, subject to guidelines and managerial approval.

• Paid work trips, including three staff retreats annually and attendance at pertinent conferences.

• Access to our well-equipped offices in Berkeley, California, which offer paid meals, snacks, a gym, and more. All employees, regardless of their location, have access to the office for at least 20 days each year.