Research Engineer

This is a fully remote position, open to applicants in Colombia.

📋 Description

• Transform vague research objectives into precise data specifications, encompassing target skills, failure modes, difficulty calibration, coverage, and success metrics.

• Develop rubrics, counterexamples, and boundary cases that establish data quality benchmarks.

• Conduct audits on generated data to identify subtle errors, reward hacking, leakage, ambiguity, inconsistent assumptions, and distribution shifts.

• Propel continuous enhancements by utilizing error taxonomies, slice-based quality metrics, and refinements informed by model behavior.

• Design and construct task suites, ground-truth signals, environment interfaces, datasets, and reinforcement learning (RL) environments.

• Execute automated validation and filtering processes, including deduplication, decontamination, leakage checks, consistency checks, and difficulty/diversity management.

• Create synthetic data generation and augmentation pipelines, featuring task generators, hard negatives, scenario templates, and simulator/tool-driven rollouts.

• Produce dataset documentation and data cards.

• Plan and conduct evaluations that mirror customer usage.

• Analyze pre/post capability slices, error breakdowns, failure narratives, and data attribute ablations.

• Conduct fine-tuning or RL-style experiments, or collaborate with research teams, to showcase measurable improvements in models.

• Work alongside engineers, researchers, quality assurance professionals, domain subject matter experts, and large-scale data production teams.

• Deliver clear specifications, examples, edge cases, quantitative feedback loops, and structured quality assessments.

• Review and enhance annotation/creation outputs while not primarily responsible for hiring, staffing, or personnel operations.

⛳️ Requirements

• 4–5 years of experience in building or enhancing deep learning systems where data quality significantly impacts outcomes.

• Strong intuition for data collection, filtering, synthesis, and measurement aimed at model enhancement.

• Ability to communicate effectively with researchers and engineers.

• Highly detail-oriented in diagnosing subtle data quality concerns and failure mechanisms.

• Proficiency in Python is mandatory.

• Familiarity with SQL/structured data workflows is strongly preferred.

• Expertise in one or more major programming languages such as C++, Java, Go, Rust, or JS/TS is advantageous for coding-oriented tasks.

• Experience with rubrics, validation scripts, gold sets, sampling strategies, statistical evaluations, slice-based assessments, and human-in-the-loop review processes.

• Experience with reinforcement learning or post-training methods is a significant advantage.

• Familiarity with agentic evaluation is a strong plus.

• Multimodal expertise is a strong plus.

• Depth in STEM disciplines is a strong plus.

• Experience with modern embodied AI/VLM-driven agents is a strong plus.

• Systems thinking and the ability to design tasks that reflect real-world constraints and workflows is a strong plus.

🏝️ Benefits

• Direct engagement with leading AI laboratories and enterprises at the forefront of post-training and RL environment development.

• Opportunity to contribute to pioneering AI research and present findings at conferences such as ICLR, ICML, and NeurIPS.

• Collaboration and knowledge sharing with outstanding colleagues from Google, Meta, Amazon, and other top-tier technology firms.

• High degree of autonomy, rapid iterations, and an exceptional learning trajectory.

• Startup-like environment offering ownership and tangible impact.

• Commitment to equal opportunity, diversity, inclusivity, and authenticity in the workplace.