
Principal AI Governance Scientist
Posted 16 hours ago

Posted 16 hours ago
This is a fully remote position, open to applicants in United States.
• Oversee technical evaluation and assurance initiatives within the AI Governance team.
• Develop enterprise-level, decision-focused methodologies for red teaming, testing, and assessing AI systems across traditional ML, Generative AI, and Agentic AI applications.
• Create reproducible frameworks to assess AI value, user impact, and broader outcomes.
• Design robust evaluation methodologies for AI performance, safety, reliability, and alignment throughout the AI lifecycle.
• Formulate criteria and benchmarks for assessing AI applications and their risk profiles.
• Plan and execute thorough red team exercises to uncover vulnerabilities, failure modes, and unintended behaviors.
• Set standards for evaluation coverage, rigor, and documentation.
• Assess business value, user impact, and societal outcomes utilizing causal inference and experimental design.
• Promote AI evaluation and safety research through white papers and training sessions.
• Offer technical leadership and mentorship to scientists, engineers, and compliance professionals.
• Identify new risks arising from advanced AI capabilities, particularly frontier models.
• Convert technical findings into actionable recommendations for leadership, governance boards, and cross-functional teams.
• Collaborate with external researchers, institutions, and industry partners.
• Undertake additional duties as assigned.
• Adhere to all policies and standards.
• Bachelor’s Degree in Computer Science, Machine Learning, Statistics, or a related quantitative field; or equivalent experience is required.
• 6+ years of experience in AI/ML research, including at least 3 years focused on model evaluation, safety, or robustness is required.
• Extensive technical expertise in modern AI systems with practical experience in evaluating large language models, generative AI, and/or agentic systems is required.
• Demonstrated history of designing rigorous evaluation methodologies and a publication record is required.
• Strong grounding in statistical methods, experimental design, causal inference, and excellent Python programming skills with ML frameworks is required.
• Candidates must be authorized to work in the U.S. without the need for employment-based visa sponsorship now or in the future.
• A Master’s Degree is preferred.
• Industry-related certifications are preferred.
• Competitive pay.
• Health insurance.
• 401K and stock purchase plans.
• Tuition reimbursement.
• Paid time off plus holidays.
• Flexible work arrangements, including remote, hybrid, field, or office schedules.
• Additional incentives may be included in the total compensation package.
• We are a diversity-focused equal opportunity employer.
CVS Health Careers
Sourcegraph
MCA Connect
Lightly
Get handpicked remote jobs straight to your inbox weekly.