
Physician, Medical AI Evaluation
Posted Nov 13

Posted Nov 13
This is a fully remote position, open to applicants in Kansas.
• Assess clinical EHR vignettes along with associated questions, suggested answers, and misleading trap answers throughout diagnosis and treatment tasks.
• Evaluate the quality of clinical reasoning, the accuracy and completeness of vignettes, correctness and gradeability of answers, whether the vignette reveals the answer, and the quality of traps.
• Determine if reasoning chains derive answers solely from the facts presented in the vignette.
• Correct reasoning pathways and compose brief rationales.
• Engage in three independent blind evaluations followed by medical professional adjudication.
• Fulfill approved tasks via the platform.
• Enhance medical AI tools through the evaluation of clinical cases.
• Medical degree (MD or DO).
• Active, unrestricted US medical license, verified at onboarding and re-verified monthly.
• Board certification or completion of residency training.
• Direct patient care experience; experience at the attending level is preferred.
• Extensive, multi-system diagnostic and treatment experience in internal medicine, family medicine, emergency medicine, or hospital medicine.
• Strong clinical reasoning skills, including differential diagnosis, next-step management, and application of evidence-based guidelines.
• Proficient written English (C1+).
• US-licensed physician residing in an eligible US state.
• Qualifications must be completed prior to joining the project.
• Not suitable for medical students, unlicensed medical graduates, non-physician clinicians, or non-clinical healthcare professionals.
• Fixed rate for each approved task, predetermined in advance.
• One-time onboarding bonus of $50, issued once the first task on the project receives approval.
• Quality incentives: additional compensation for high-quality approved tasks.
• Fully self-scheduled work with no minimum hour requirements, established schedule, or mandatory availability.
• Opportunity for remote work.
• No patient interaction and no risk of malpractice exposure.
• Participation is limited to this project.
• Flexible, time-sensitive project that can accommodate other commitments.
CVS Health Careers
Sourcegraph
MCA Connect
Lightly
Get handpicked remote jobs straight to your inbox weekly.