Field Engineer, AI Labs

This is a fully remote position, open to applicants in California, +2 more states.

📋 Description

• Integrate frontier model APIs and inference endpoints into evaluation frameworks.

• Conduct extensive benchmark evaluations utilizing proprietary Turing datasets focused on reasoning, coding, multimodal environments, agentic settings, and GDP Val.

• Examine evaluation outcomes, identify failure points, observe model behavioral shifts, and pinpoint necessary data distributions to bridge capability gaps.

• Act as the technical liaison for sales leads during discussions with frontier AI laboratories and enterprise AI teams.

• Convert evaluation metrics into reports, client presentations, and technical documentation.

• Respond to technical inquiries and RFIs regarding data design, task curation, difficulty stratification, and evaluation rubrics.

• Communicate client feedback and emerging benchmarking needs to internal R&D and dataset development teams.

• Manage Turing's internal data infrastructure, which includes DataOS, Data Hub, and task catalogs.

• Query, manipulate, and transfer data pipelines.

• Create and curate customized data subsets based on difficulty levels, benchmark rubrics, or top-performing data points.

• Audit data provenance and uphold specifications across both public catalogs and internal dataset inventories.

• Oversee the final delivery, schema alignment, and packaging of client data submissions.

• Collaborate with engineering and quality leads to ensure deliverables are validated prior to client handoff.

⛳️ Requirements

• 1 to 5 years of professional experience in data science, AI/ML engineering, solutions engineering, or data analytics, showcasing a transition into generative AI.

• Strong practical proficiency in Python.

• Experience with REST/gRPC model APIs and inference endpoints, including OpenAI, Anthropic, Bedrock, and vLLM.

• Familiarity with pandas and SQL.

• Hands-on experience with model evaluation, benchmark development, or LLM-as-a-judge pipelines.

• Knowledge of benchmark structure, scoring methodologies, and debugging techniques.

• Proven ability to manage projects from inception to completion and meet tight deadlines.

• Outstanding written and verbal communication skills.

• Capability to elucidate technical evaluation metrics to research scientists and engage confidently in commercial discussions.

• Comfort with command-line tools, internal software, and data manipulation processes.

• Preferred: experience at an early-stage AI startup or a rapidly growing technology company.

• Preferred: familiarity with evaluation frameworks and agent benchmarks such as Harbor, Terminal-Bench, SWE-bench, or Inspect.

• Preferred: experience in fine-tuning models, utilizing open-weight models, or conducting multimodal evaluations.

• Preferred: knowledge of data quality control, task verification, or synthetic data generation processes.

🏝️ Benefits

• Equity.

• Performance incentives.

• Comprehensive company benefits.

• Opportunity to work at the forefront of AI innovation.

• Chance to contribute to cutting-edge AI research and showcase work at ICLR, ICML, and NeurIPS.

• Opportunity to bring frontier AI advancements to enterprise solutions.

• Collaboration and learning opportunities with colleagues possessing extensive AI expertise.

• Startup-like pace, ownership, and impactful contributions.