Senior Research Infrastructure Engineer

This is a fully remote position, open to applicants in California.

📋 Description

• Develop infrastructure to support research on agent behavior, ensuring reliable environment execution, trajectory collection, and large-scale evaluation.

• Create reusable tools, APIs, and pipelines for conducting experiments, managing datasets and configurations, as well as inspecting trajectories and evaluation outcomes.

• Transform research prototypes and experimental processes into thoroughly tested, documented, and reusable software solutions.

• Design experiments aimed at understanding model behavior, identifying agent failure modes, and addressing data quality challenges.

• Enhance the speed, reproducibility, and cost-effectiveness of experimentation via orchestration, observability, and recovery strategies for failed runs.

• Collaborate with research and engineering teams to align research requirements with infrastructure priorities and architectural decisions.

• Take ownership of systems from their implementation through to ongoing operation.

• Accelerate research efforts focused on training and evaluating cutting-edge AI agents.

⛳️ Requirements

• Strong foundation in software engineering principles.

• Proficiency in Python programming.

• Proven track record of developing maintainable systems that are utilized by others.

• Experience managing backend services, data pipelines, developer tools, or infrastructure from conception to deployment and operation.

• Good judgment regarding APIs, data models, asynchronous workloads, and reliability trade-offs.

• Familiarity with Docker, Linux, cloud infrastructure, and troubleshooting failures across application code and systems.

• Ability to collaborate closely with researchers and convert open-ended research needs into practical software solutions.

• Independent technical judgment for prioritizing tasks and making pragmatic architectural decisions.

• Working knowledge of reinforcement learning concepts, such as environments, rollouts, reward signals, and evaluation workflows is a plus.

• Experience in building research infrastructure, ML pipelines, or tools aimed at enhancing research productivity is advantageous.

• Understanding of distributed job execution, containerized environments, or high-concurrency workloads is beneficial.

• Experience in enhancing experiment reproducibility, data lineage, or pipeline failure visibility can be advantageous.

• Technical skills and learning potential are prioritized over the number of years of experience.

🏝️ Benefits

• Comprehensive medical, dental, and vision coverage at 100% for employees in the US and Singapore.

• Complimentary lunch and dinner for employees working in the office.

• Company-wide holiday break from Christmas Eve to New Year’s Day, in addition to PTO and paid holidays.

• 401k plan with a 4% matching contribution.

• Equinox gym membership provided for employees in the US.

• Commuter benefits available for US employees.

• Health and wellness stipend for employees in Singapore.

• Unlimited access to tokens for tools such as ChatGPT, Claude Code, Cursor, etc.

• Annual travel budget for visits to either office location.

• Relocation and visa assistance for qualified full-time candidates moving to the US or Singapore.