Research Engineer, Frontier Data

This is a fully remote position, open to applicants in Brazil, +1 more country.

📋 Description

• Translate the evolving research objectives of clients into technical proposals and precise data requirements.

• Provide assessments of feasibility, key assumptions, criteria for acceptance, and evaluation requirements.

• Develop rubrics, counterexamples, and boundary cases.

• Perform audits on generated data to identify subtle errors, instances of reward hacking, leakage, ambiguity, inconsistent assumptions, and shifts in distribution.

• Categorize failures and transform findings into actionable production instructions and enhancements.

• Design task suites, ground-truth signals, verifiers, unit tests, reward functions, validators, environment interfaces, APIs, tool schemas, state abstractions, and database schemas.

• Construct project-specific verification systems that incorporate deterministic checks, model-based evaluators, and human reviews.

• Implement automated validation and filtering mechanisms, including deduplication, decontamination, checks for leakage and consistency, and controls for difficulty/diversity.

• Develop pipelines for synthetic data generation and augmentation.

• Prepare dataset documentation and data cards.

• Design and conduct evaluations that reflect client usage.

• Analyze pre/post comparisons, error breakdowns, and ablations to correlate data with model outcomes.

• Carry out fine-tuning or reinforcement learning-style experiments when necessary to demonstrate quantifiable model enhancements.

• Provide technical leadership to distributed expert data-production networks.

• Oversee technical translation, acceptance criteria, evaluator/verifier design, quality diagnosis, and recommendations for technical releases.

• Communicate technical insights, quality risks, and recovery strategies directly to clients.

• Primarily collaborate with a single frontier AI lab client while also supporting additional accounts as required.

⛳️ Requirements

• 4–5 years of experience in building or enhancing deep learning systems where data quality is critically important.

• Strong intuition for data collection, filtering, synthesis, and measurement aimed at model improvement.

• Ability to translate research objectives into concrete technical specifications and convert outputs into actionable insights.

• Comfortable acting as the primary technical contact for clients.

• Highly detail-oriented approach to diagnosing subtle data quality issues and failure modes.

• Proficiency in Python is required.

• Familiarity with SQL/structured data workflows is strongly preferred.

• For coding-focused tasks, proficiency in a major programming language such as C++, Java, Go, Rust, or JS/TS is a plus.

• Experience in designing rubrics, validation scripts, gold sets, sampling strategies, statistical checks, slice-based evaluations, and human-in-the-loop review processes.

• Experience with reinforcement learning or post-training methodologies is a strong advantage, including RLHF/RLAIF, verifier training, reward modeling, RL fine-tuning, or environment design.

• Experience in agentic evaluation is a strong plus.

• Multimodal expertise is a strong plus.

• A solid background in STEM fields such as math, physics, or engineering is a strong plus.

• Experience with modern embodied AI/VLM-driven agents is a strong plus.

• Systems-thinking capability to simulate application APIs/data schemas and design realistic tasks.

• Direct experience with frontier AI labs is not a requirement.

🏝️ Benefits

• Work directly with a leading frontier AI lab at the forefront of post-training and reinforcement learning environment design.

• Enjoy high autonomy and rapid iteration.

• Be part of a talent-dense team that offers a steep learning curve.

• Opportunity to present your work at prestigious conferences such as ICLR, ICML, and NeurIPS.

• Collaborate with and learn from exceptional colleagues with extensive AI expertise.

• Thrive in a startup-speed environment where you can make a significant impact.

• Benefit from an equal opportunity, diverse, inclusive, and authentic workplace.