
Physician Reviewer, MD/DO β Medical AI Evaluation
Posted Nov 13

Posted Nov 13
This is a fully remote position, open to applicants in Wisconsin.
β’ Evaluate clinical EHR vignettes that include questions, suggested answers, and misleading trap answers.
β’ Assess tasks related to diagnosis and treatment across various organ systems.
β’ Score the accuracy and completeness of vignettes, correctness and gradeability of answers, quality of reasoning, and effectiveness of traps.
β’ Determine if reasoning chains derive answers solely from the facts presented in the vignettes.
β’ Correct reasoning pathways and compose brief rationales.
β’ Engage in three independent blind evaluations followed by adjudication by physicians.
β’ Fulfill qualification tasks and project assignments via the platform.
β’ Submit tasks for approval and receive compensation for work that has been approved.
β’ Possession of a medical degree (MD or DO).
β’ Active, unrestricted medical license in the US, verified during onboarding and re-checked monthly.
β’ Board certification or completion of residency training.
β’ Experience in direct patient care; attending-level experience preferred.
β’ Extensive diagnostic and treatment experience across multiple systems in internal medicine, family medicine, emergency medicine, or hospital medicine.
β’ Strong clinical reasoning skills, including differential diagnosis, next-step management, and adherence to evidence-based guidelines.
β’ Proficient written English skills (C1+ level).
β’ Prior experience in medical AI evaluation, clinical content review, exam question review, or medical annotation/QA is an advantage.
β’ Not suitable for medical students, unlicensed medical graduates, non-physician clinicians, or individuals with non-clinical healthcare titles.
β’ Compensation per approved task at a predetermined fixed rate.
β’ Flexible self-scheduling with no minimum hour requirements or set schedule.
β’ Opportunity for remote work.
β’ No patient interaction and no exposure to malpractice risks.
β’ One-time onboarding reward of $50, payable upon approval of the first task.
β’ Quality incentives for tasks that receive high-quality approvals.
β’ Flexible participation that accommodates other commitments.
CVS Health Careers
Sourcegraph
MCA Connect
Lightly
Get handpicked remote jobs straight to your inbox weekly.