Research Engineer / Scientist – Reinforcement Learning

This is a fully remote position, open to applicants in California.

📋 Description

• Design and assess reinforcement learning (RL) techniques for reasoning, coding, and long-term tasks, including modifications to training objectives and policy optimization.

• Explore stability issues in asynchronous RL, focusing on policy staleness, discrepancies between trainers and samplers, and numerical variations between training and rollout phases.

• Collaborate with data and evaluation researchers to gain insights on how task selection, rewards, and training datasets influence model learning.

• Create experiments to analyze how RL performance and learning dynamics are affected by model size, computational resources for training, and the duration of tasks.

• Implement controlled ablations and evaluations to differentiate between genuine learning improvements and artifacts such as reward exploitation, evaluation inconsistencies, or training errors.

• Transform experimental insights into training methodologies and verify whether improvements are consistent across larger models and extended runs.

• Enhance prime-rl to accommodate new algorithms, models, and asynchronous training workflows.

• Collaborate with the infrastructure team to boost performance and numerical precision of research runs across various trainers, rollout workers, and environments.

• Develop diagnostic tools that facilitate the reproduction of unstable runs and performance regressions, aiding in the differentiation of algorithmic issues from implementation or runtime challenges.

⛳️ Requirements

• Proven expertise in at least one of the following areas: RL algorithms, empirical research on model training, or distributed machine learning systems.

• Proficient programming capabilities in Python and familiarity with PyTorch or a similar machine learning framework.

• Ability to design controlled experiments, analyze results, and troubleshoot unexpected model or system behaviors.

• Experience in taking a research concept or system modification from initial implementation through to testing and evaluation, with a detailed account of successes and challenges.

• Familiarity with distributed training frameworks or asynchronous rollout processes.

• Expertise in diagnosing unstable RL runs, including issues related to policy staleness or numerical inaccuracies.

• Experience in developing coding agents, verifiable rewards, or long-term environment simulations.

🏝️ Benefits

• Equity incentives.

• Flexible work arrangements, including the possibility to work remotely or in-person at our San Francisco offices.

• Visa sponsorship and relocation assistance available for international applicants.

• Quarterly team off-site events.

• Opportunities for hackathons.

• Access to conferences and learning opportunities.

• Chance to collaborate with a talented, dedicated, and mission-driven team.