ML Engineer, Agentic Systems

This is a fully remote position, open to applicants in North America.

📋 Description

• Set the strategic direction for the Code Understanding team regarding models, evaluations, and agentic systems.

• Design and refine multi-step, tool-utilizing agent loops to create reliable, observable, and cost-effective enterprise-scale products.

• Decide the appropriate times and methods for implementing evaluations, smoke tests, metrics, and qualitative assessments.

• Choose, enhance, fine-tune, and train models when necessary.

• Enhance retrieval, ranking, context windows, and citations for code-grounded solutions.

• Optimize model cost and latency through profiling, distillation, caching, and proper sizing.

• Take full ownership of significant agentic product segments from problem identification to rollout and evaluation.

• Develop evaluations, dashboards, and safeguards for responsible model and prompt modifications.

• Mentor and elevate team members in agent engineering.

• Collaborate with customers to translate their feedback into requirements, scopes, and milestones.

• Contribute to the codebase and influence technical decisions beyond the immediate team.

• Participate in the on-call support rotation.

• Lead roadmap initiatives and drive measurable enhancements in answer quality, cost, latency, and agentic capabilities.

⛳️ Requirements

• Staff engineer and technical leader with expertise in production machine learning, evaluation, and agent systems.

• Have personally overseen a production model lifecycle from dataset creation through evaluation, rollout, and monitoring.

• Experience in training or fine-tuning at least one model.

• Proven track record in designing reliable, observable, and cost-efficient multi-step agentic systems.

• Strong evaluation judgment, including representative datasets, baselines, error classifications, and release criteria.

• Ability to navigate quality, latency, and cost trade-offs through model selection, prompting, retrieval, caching, distillation, and fine-tuning.

• Capability to work independently on ambiguous, high-technical-risk challenges.

• Strong software engineering skills with the ability to deploy production services.

• Proficient in Go, TypeScript, GraphQL, Postgres, and Docker, or eager to learn these technologies.

• Familiarity with agentic coding tools and a thorough understanding of every line of code submitted.

• Comfortable in a fast-paced, remote environment that prioritizes asynchronous communication and operates with multiple services.

• Ability to mentor engineers through pairing, design reviews, and code reviews.

• A customer- and product-focused mindset.

• Availability to work overlapping hours with EST for at least 20 hours per week.

🏝️ Benefits

• Meaningful equity.

• Competitive cash compensation.

• Generous perks and benefits.

• Open and transparent compensation philosophy.

• Pay bands established for competitive and equitable compensation.

• Globally distributed remote work arrangement.

• Flexible location options available in nearly any part of the world.

• Equal opportunity workplace.