AI Associate – MD

📋 Description

• Validation of AI clinical tools and enhancement of LLM capabilities.

• Oversee pre-deployment validation processes.

• Create model cards for transparency, detailing intended use, limitations, monitoring, and rollback procedures.

• Collaborate on ensuring data quality, focusing on provenance, representativeness, refresh cadence, and privacy considerations.

• Develop evaluation datasets and gold standards for LLMs; facilitate clinical red-teaming and prompt testing; refine prompts and guardrails; evaluate RAG fidelity.

• Enable clinical-in-the-loop processes: Train clinicians to manage AI tools and handle exceptions; formulate SOPs, checklists, and escalation protocols for LLM usage.

• Work in partnership with engineering, product, security, compliance, legal, and care/service line leaders.

• Manage AI governance and policy: Oversee clinical AI governance concerning ethics, safety, regulation, privacy, change control, and documentation in collaboration with the Patient Safety Officer.

• Establish criteria for clinical use and human-in-the-loop guardrails.

• Maintain governance documentation, including charters, SOPs, and decision logs, ensuring audit readiness.

• Promote quality and patient safety: Support clinical QA activities; engage in URAC/NCQA initiatives; manage policies, standards, and rubrics.

• Lead or participate in event detection, RCA/FMEA, CAPA processes; provide education through CME/CE and targeted training.

• Assist with client and payer audits, RFPs, VBC metrics (HEDIS, MIPS), implementations, and handling escalations, grievances, and appeals.

• Lead or assist cross-functional teams in analyzing clinical quality performance data, identifying improvement opportunities, and translating insights into enhancements for health plan design that optimize outcomes, member experience, and cost efficiency.

• Conduct continuous assessments of clinical services, applying improvement methodologies and evidence-based practices to ensure effectiveness, measurable patient outcomes, and compliance with organizational quality standards.

⛳️ Requirements

• MD/DO with an active, unrestricted medical license.

• A minimum of 3 years in clinical practice; at least 2 years in quality, patient safety, or clinical operations.

• Experience in the clinical review of LLM outputs; developing evaluation datasets; conducting red-teaming and prompt testing; and assessing RAG fidelity.

• Expertise in quality improvement and patient safety methodologies (PDSA, Lean/Six Sigma, RCA/RCA2, FMEA, CAPA).

• Familiarity with URAC, NCQA, and CMS; knowledge of HEDIS/MIPS is a plus.

• A proven history of successfully implementing quality improvements within value-based care frameworks.

• Experience collaborating with analytics and data science teams; ability to interpret core performance metrics.

• Strong documentation skills and effective communication with stakeholders.

• Preferred experience in building, deploying, and refining clinical AI agents.

• Preferred proficiency in business intelligence and analytics tools (e.g., Tableau/Power BI; basic SQL/statistics).

• Preferred experience in virtual care, care navigation, and conducting enterprise audits.

🏝️ Benefits

• Remote-first work culture.

• 401(k) savings plan managed through Fidelity.

• Extensive medical, vision, and dental coverage with various medical plan options, including disability insurance.

• Paid Time Off ("PTO") and Discretionary Time Off ("DTO").

• 12 weeks of fully paid parental leave.

• Family Building & Compassionate Leave: Fertility coverage, $25,000 for surrogacy/adoption, and paid leave for unsuccessful treatments, adoption, or pregnancies.

• Reimbursement for work-from-home expenses to facilitate collaboration in a home office setting.