
Research Engineering Manager, STEM
Posted 2 days ago

Posted 2 days ago
This is a fully remote position, open to applicants in Brazil, +1 more country.
• Lead and develop a team of research engineers.
• Define team roles, conduct hiring, and nurture engineers with a focus on research acumen and engineering excellence.
• Oversee project selection, planning, experiment evaluations, design documentation, ownership responsibilities, and performance cycles.
• Mentor engineers on experimental design, code quality, and the presentation of results.
• Establish standards for evaluation methodologies, data quality, and reproducibility.
• Collaborate with research scientists to formulate experiment plans, establish baselines, conduct ablation studies, and set decision criteria.
• Set the research agenda and introduce original hypotheses.
• Transition research prototypes to production scale.
• Manage synthetic data pipelines for generation, verification, filtering, deduplication, and version management.
• Assess the impact of synthetic data through ablation studies and fine-tuning or reinforcement learning experiments.
• Construct quality systems utilizing gold sets, LLM-judge calibration, sampling audits, and agreement metrics.
• Design and deploy new evaluations and benchmarks featuring calibrated difficulty, contamination controls, held-out splits, and confidence intervals.
• Develop deterministic, reproducible evaluation harnesses, including agentic and sandboxed-execution assessments.
• Validate the reliability of benchmark results.
• Publish benchmarks, datasets, and findings in collaboration with research scientists.
• Allocate a third or more of the time to hands-on technical work, including verifier logic, evaluation statistics, and pipeline components.
• Act as a technical liaison to cutting-edge AI laboratories.
• Collaborate with Strategic Project Leads, delivery teams, and engineering personnel to integrate research into client projects and platforms.
• Over 8 years of practical experience in machine learning or research engineering.
• Minimum of 5 years in an industry setting.
• At least 3 years of direct management experience with research engineers, ML engineers, or researchers, encompassing hiring and performance evaluation.
• Proven experience managing synthetic or curated data pipelines in production that influenced training decisions.
• Demonstrated ability to design and deliver LLM evaluations or benchmarks that inform model decisions.
• Knowledge of contamination, saturation, judge bias, inter-annotator agreement, variance, and confidence intervals.
• Proficient in Python.
• Production experience with PyTorch or JAX.
• Familiarity with SFT, DPO, PPO/GRPO, and reinforcement learning with verifiable rewards.
• Experience in distributed data processing and handling large evaluation or training tasks, such as Ray, Spark, or Kubernetes, on AWS or GCP.
• Regular use of LLM coding tools like Claude Code, Cursor, or Copilot is essential.
• A PhD or Master's degree in AI, machine learning, computer science, statistics, or a closely related discipline; a PhD is preferred.
• Exceptional equivalent research experience may also be considered.
• Strong written communication skills and the ability to effectively present results to partner-lab researchers.
• Engage directly with leading AI laboratories and enterprises at the forefront of post-training, evaluation, and synthetic data.
• Develop datasets and benchmarks that enhance and assess advanced AI systems.
• Enjoy high autonomy, rapid iteration, and significant commercial impact.
• Collaborate with outstanding colleagues from top technology firms.
• Contribute to research disseminated through technical publications and prominent conferences.
• Work in an equal opportunity, diverse, inclusive, and authentic environment.
Medline Industries, LP
BeOne Medicines
SANS Institute
Spotnana
Get handpicked remote jobs straight to your inbox weekly.