Applied AI Scientist, Clinical AI Agents

This is a fully remote position, open to applicants in New York.

📋 Description

• Design, develop, and enhance agentic AI systems tailored for intricate healthcare workflows, such as documentation, coding, denial management, appeals, and revenue cycle automation.

• Create long-term agent behaviors encompassing context construction, retrieval, tool usage, memory, routing, verification, escalation, and human-in-the-loop review.

• Establish clear criteria for what constitutes “good” performance for clinical agents from end to end, transforming expert workflows into specifications, rubrics, gold standards, test cases, and clinically relevant success metrics.

• Construct robust evaluation and feedback mechanisms utilizing expert reviews, production logs, model outputs, and benchmarks to assess performance, regressions, edge cases, safety, reliability, provenance quality, and business impact.

• Prototype innovative AI capabilities from initial concept to reliable, explainable, and auditable production systems, complete with clear contracts, monitoring, evidence, rationale, and performance thresholds.

• Collaborate with research and ML engineering teams on model selection, fine-tuning, reward modeling, distillation, synthetic data, post-training, and internal AI infrastructure, including instrumentation, experiment tracking, benchmarking, prompt/version management, and reproducible evaluation.

⛳️ Requirements

• Possess 4+ years of experience in software engineering, ML engineering, research engineering, or applied AI.

• Demonstrate strong proficiency in Python and have the capability to build production systems utilizing APIs, structured data, asynchronous workflows, testing, logging, and observability.

• Have a background in transforming disorganized real-world workflows into structured AI challenges, including classification, ranking, extraction, decision-making, LLM applications, agents, RAG, tool invocation, structured outputs, prompting, or evaluation.

• Experience in developing or managing evaluation systems, benchmarks, annotation workflows, experiment tracking, or regression testing for AI systems.

• Excel in ambiguous, high-stakes environments: collaborating with experts, troubleshooting real-world failures, and converting model potential into dependable, accurate, and safe systems that benefit users.

🏝️ Benefits

• Top-of-market compensation (salary + equity)

• Flexible PTO

• Comprehensive health benefits

• 401(k) matching

• Inspiring, brilliant, mission-driven teammates