Senior Applied AI Scientist, Clinical AI Agents

This is a fully remote position, open to applicants in New York.

📋 Description

• Design, construct, and enhance agentic AI systems tailored for intricate healthcare workflows, encompassing documentation, coding, denial management, appeals, and revenue cycle automation.

• Develop long-term agent behavior, focusing on context construction, retrieval, tool utilization, memory, routing, verification, escalation, and human-in-the-loop reviews.

• Establish criteria for what constitutes “good” for clinical agents from start to finish, translating expert workflows into specifications, rubrics, gold standards, test cases, and clinically relevant success metrics.

• Create robust evaluation and feedback mechanisms utilizing expert reviews, production logs, model outputs, and benchmarks to assess performance, regressions, edge cases, safety, reliability, provenance quality, and business impact.

• Prototype innovative AI capabilities from inception to maturity, subsequently solidifying them into dependable, explainable, and auditable production systems with defined contracts, monitoring, evidence, rationale, and performance thresholds.

• Collaborate with research and ML engineering teams on model selection, fine-tuning, reward modeling, distillation, synthetic data, post-training efforts, and internal AI infrastructure, which includes instrumentation, experiment tracking, benchmarking, prompt/version management, and reproducible evaluation.

⛳️ Requirements

• Possess 8+ years of experience in software engineering, ML engineering, research engineering, or applied AI.

• Exhibit strong proficiency in Python and have the capability to build production systems using APIs, structured data, asynchronous workflows, testing, logging, and observability.

• Have experience transforming chaotic real-world workflows into structured AI challenges, including classification, ranking, extraction, decision-making, LLM applications, agents, RAG, tool invocation, structured outputs, prompting, or evaluation.

• Have developed or operated evaluation systems, benchmarks, annotation workflows, experiment tracking systems, or regression tests for AI applications.

• Excel in ambiguous, high-stakes environments: collaborating with experts, troubleshooting real-world issues, and converting model potential into trustworthy, accurate, and safe systems that benefit users.

🏝️ Benefits

• Competitive compensation package (salary + equity)

• Flexible paid time off

• Comprehensive healthcare benefits

• 401(k) matching

• A team of inspiring, brilliant, mission-driven colleagues