
Physician, Medical AI Evaluation
Posted 6 days ago

Posted 6 days ago
This is a fully remote position, open to applicants in United States.
β’ Assess clinical EHR vignettes alongside questions, suggested answers, and misleading distractor options related to diagnosis and treatment tasks.
β’ Evaluate the quality of clinical reasoning, the accuracy and completeness of vignettes, the correctness and gradeability of answers, and the quality of distractors.
β’ Determine if reasoning chains derive answers solely from the facts presented in the vignettes.
β’ Rectify reasoning pathways and compose concise rationales.
β’ Engage in three independent blind evaluations followed by physician adjudication.
β’ Undertake flexible, task-oriented project work via the platform.
β’ Medical degree (MD or DO) is required.
β’ Must possess an active, unrestricted US medical license, verified at onboarding and re-verified monthly.
β’ Board certification or completion of residency training is necessary.
β’ Experience in direct patient care is required; attending-level experience is preferred.
β’ Extensive diagnostic and treatment experience across multiple systems in internal, family, emergency, or hospital medicine.
β’ Strong clinical reasoning skills, including differential diagnosis, next-step management, and the application of evidence-based guidelines.
β’ Proficient written English (C1+ level).
β’ Previous experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA is advantageous.
β’ This position is not suitable for medical students, unlicensed medical graduates, non-physician clinicians, or non-clinical healthcare titles.
β’ Compensation per approved task: a predetermined rate for each task established in advance.
β’ Fully self-scheduled: no minimum hours required and no fixed schedule.
β’ Opportunity for remote work.
β’ No patient interaction and no malpractice risk.
β’ One-time onboarding bonus of $50, paid after the approval of your first task on the project.
β’ Quality incentives: additional earnings for tasks of high quality that are approved.
β’ No requirement for time tracking or hour logging.
β’ No tiers or speed adjustments to the compensation rate.
CVS Health Careers
Sourcegraph
MCA Connect
Lightly
Get handpicked remote jobs straight to your inbox weekly.