
Physician, Medical AI Evaluation
Posted 6 days ago

Posted 6 days ago
This is a fully remote position, open to applicants in United States.
β’ Assess clinical EHR vignettes alongside associated questions, suggested answers, and distractor options related to diagnosis and treatment tasks.
β’ Evaluate the quality of clinical reasoning, the accuracy and completeness of vignettes, the correctness and gradeability of answers, and the quality of distractors.
β’ Determine if the reasoning leads to the answer based solely on the facts presented in the vignette.
β’ Correct reasoning pathways and produce concise rationales.
β’ Engage in a three-independent-blind-read process, followed by adjudication by a physician.
β’ Fulfill qualifications, join the project, submit tasks via the platform, and complete approved tasks.
β’ Select which tasks to undertake and decide when to participate.
β’ Possession of a medical degree (MD or DO).
β’ A valid, unrestricted US medical license, verified upon onboarding and re-checked monthly.
β’ Board certification or completion of residency training.
β’ Experience in direct patient care; attending-level experience is preferred.
β’ Extensive, multi-system diagnostic and treatment experience in internal medicine, family medicine, emergency medicine, or hospital medicine.
β’ Strong clinical reasoning skills, including differential diagnosis, next-step management, and adherence to evidence-based guidelines.
β’ Proficient written English (C1+ level).
β’ Previous experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA is advantageous.
β’ Not appropriate for medical students, unlicensed medical graduates, non-physician clinicians, or non-clinical healthcare professionals.
β’ Compensation per approved task: a predetermined fixed rate for each task.
β’ Fully self-scheduled: no minimum hours required and no fixed timetable.
β’ Remote working opportunity.
β’ No patient interaction and zero malpractice liability.
β’ A one-time onboarding reward of $50, payable once your initial task on the project is approved.
β’ Quality incentives: additional earnings for tasks of high quality that are approved.
β’ Flexible participation in the project with time limitations.
β’ No time tracking or hour logging required.
CVS Health Careers
Sourcegraph
MCA Connect
Lightly
Get handpicked remote jobs straight to your inbox weekly.