Lead Research Engineer, Data Quality

This is a fully remote position, open to applicants in United States, +2 more locations.

📋 Description

• Oversee HUD’s data quality strategy, which includes establishing QC systems, defining and enforcing quality standards, and designing experiments to evaluate agent outputs.

• Create methods for validating synthetic data at scale, encompassing failure-mode analysis, task mutation checks, and trajectory auditing.

• Collaborate with research engineers, domain specialists, and data vendors to identify quality issues and enhance data generation workflows.

• Transform qualitative research findings into production systems, internal tools, dashboards, validation pipelines, and feedback mechanisms.

• Develop an internal research understanding of what constitutes effective agent training data, focusing on utility beyond superficial correctness.

• Guide research engineers in upholding high standards for technical rigor, clarity, and execution efficiency.

• Direct the data quality team in assessing thousands of tasks across RL environments, synthetic data, benchmarks, and domain-specific workflows.

⛳️ Requirements

• Advanced expertise in Python, Docker, and Linux environments.

• In-depth understanding of data quality, including the characteristics that render tasks realistic, learnable, diverse, reliable, and beneficial for training.

• Proven experience in building QC systems, evaluations, benchmarks, synthetic data pipelines, validation workflows, or model evaluation infrastructure.

• Proficiency in navigating complex human and technical systems, including collaboration with domain experts, vendors, generated data, model outputs, graders, and infrastructure.

• Exceptional written communication skills and the ability to articulate methodology clearly to researchers, engineers, laboratories, and external audiences.

• Experience leading teams on ambiguous technical projects from problem definition through to implementation and iteration.

• Proven ability to work with subject-matter experts to capture domain expertise and translate it into scalable review or generation systems.

• Competence in designing metrics, experiments, and QA/QC processes.

• Experience in early-stage startups and the capacity to work independently in fast-paced settings.

• Detail-oriented with a keen eye for identifying subtle inconsistencies or edge cases in data.

• Ability to maintain working hours that overlap 70–80% with San Francisco or Singapore time zones.

🏝️ Benefits

• Comprehensive coverage for top-tier medical, dental, and vision plans from Blue Shield of CA (for US employees).

• Complimentary lunch and dinner for in-office employees.

• Company-wide holiday break from Christmas Eve to New Year’s Day, in addition to PTO and paid holidays.

• Equinox membership provided for US employees.

• 401k plan available for US employees.

• Commuter benefits offered for US employees.

• Unlimited access to tokens for ChatGPT, Claude Code, Cursor, etc.

• Assistance with relocation and visa applications for strong full-time candidates to the US or Singapore.