
Physician, MD/DO – Put Medical AI to the Test
Posted 4 days ago

Posted 4 days ago
This is a fully remote position, open to applicants in Alaska.
• Assess clinical EHR vignettes alongside associated questions, suggested answers, and misleading distractor answers related to diagnosis and treatment tasks.
• Evaluate the quality of clinical reasoning, the accuracy and completeness of vignettes, the correctness and gradeability of answers, and whether the vignette discloses the answer, as well as the quality of traps.
• Determine if reasoning chains derive answers solely from the facts presented in the vignette.
• Amend reasoning traces and compose concise rationales.
• Conduct three independent blind evaluations for each case, followed by physician adjudication.
• Fulfill qualifications, engage in the project, submit tasks via the platform, and complete the tasks that receive approval.
• Possession of a medical degree (MD or DO).
• An active, unrestricted medical license in the US, verified during onboarding and re-checked monthly.
• Board certification or completion of residency training.
• Experience in direct patient care; attending-level experience is preferred.
• Extensive diagnostic and treatment experience across multiple systems in internal, family, emergency, or hospital medicine.
• Strong clinical reasoning skills, including differential diagnosis, next-step management, and the application of evidence-based guidelines.
• Proficient written English (C1+ level).
• Previous experience in medical AI evaluation, clinical content or exam-question review, or medical annotation/QA is advantageous.
• Status as a US-licensed physician; this role is not suitable for medical students, unlicensed medical graduates, non-physician clinicians, or those with non-clinical healthcare titles.
• Compensation per approved task: a fixed rate established in advance for each task.
• Fully self-scheduled: no minimum hour requirements and no set schedule.
• Opportunity for remote work.
• No patient interaction and no malpractice liability.
• A one-time onboarding bonus of $50, payable once the first approved task on the project is completed.
• Quality incentives: additional compensation for high-quality approved tasks.
• Flexible participation around other obligations.
• No requirements for time tracking or hour logging.
• Increased per-task rates for QA and adjudication tasks.
RWS Group
Stripe
Mindrift
Mindrift
Get handpicked remote jobs straight to your inbox weekly.