
Physician Reviewer, MD/DO β Medical AI Evaluation
Posted 4 days ago

Posted 4 days ago
This is a fully remote position, open to applicants in New York.
β’ Assess clinical EHR vignettes in conjunction with questions, suggested answers, and distractor βtrapβ answers related to diagnosis and treatment tasks.
β’ Evaluate benchmark items based on vignette accuracy and completeness, answer disclosure, correctness of answers, gradeability, and trap quality.
β’ Verify if reasoning chains lead to answers solely based on vignette facts.
β’ Adjust reasoning pathways and compose concise rationales.
β’ Engage in three independent blind readings followed by physician adjudication.
β’ Fulfill qualifications, join the project, submit tasks via the platform, and complete assigned tasks that have been approved.
β’ Enhance medical AI tools through detailed clinical case evaluations.
β’ Medical degree (MD or DO).
β’ Active, unrestricted US medical license, verified at onboarding and re-verified monthly.
β’ Board certification or completion of residency training.
β’ Experience in direct patient care; attending-level experience is preferred.
β’ Extensive, multi-system diagnostic and treatment experience in internal, family, emergency, or hospital medicine.
β’ Strong clinical reasoning skills, including differential diagnosis, next-step management, and the application of evidence-based guidelines.
β’ Proficient in written English (C1+).
β’ Prior experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA is advantageous.
β’ Must be a US-licensed physician residing in an eligible US state.
β’ Not suitable for medical students, unlicensed medical graduates, non-physician clinicians, or those with non-clinical healthcare titles.
β’ Compensation per approved task at a predetermined fixed rate.
β’ Completely self-scheduled work with no minimum hours or established schedule.
β’ Opportunity for remote work.
β’ No patient contact involved.
β’ No exposure to malpractice risks.
β’ One-time onboarding bonus of $50, paid after the first task on the project is approved.
β’ Quality incentives: additional earnings for tasks approved with high quality.
β’ Flexible, time-limited project-based participation.
β’ Increased per-task rates for QA and adjudication tasks.
CVS Health
RWS Group
RWS Group
RWS Group
Get handpicked remote jobs straight to your inbox weekly.