Physician, MD/DO – Medical AI Evaluation

This is a fully remote position, open to applicants in Utah.

📋 Description

• Assess clinical EHR vignettes alongside questions, suggested answers, and misleading “trap” answers related to diagnosis and treatment tasks.

• Evaluate benchmark items for the accuracy and completeness of vignettes, answer disclosure, correctness and gradeability of answers, as well as the quality of traps.

• Determine if reasoning chains derive answers solely from the facts presented in the vignettes.

• Revise reasoning traces and compose brief rationales.

• Engage in three independent blind reviews followed by adjudication by a physician.

• Submit tasks via the platform for assessment and approval based on project acceptance standards.

• Select tasks to undertake and the timing of contributions.

• Enhance medical AI tools utilized by healthcare professionals.

⛳️ Requirements

• Medical degree (MD or DO).

• Active, unrestricted US medical license, verified during onboarding and re-verified monthly.

• Board certification or completion of residency training.

• Experience in direct patient care; attending-level experience preferred.

• Extensive diagnostic and treatment experience across multiple systems in internal, family, emergency, or hospital medicine.

• Excellent clinical reasoning skills, encompassing differential diagnosis, next-step management, and the application of evidence-based guidelines.

• Proficient written English skills (C1+).

• Previous experience in medical AI assessment, clinical content or exam-question review, or medical annotation/QA is advantageous.

• Not suitable for medical students, unlicensed medical graduates, non-physician clinicians, or individuals holding non-clinical healthcare titles.

• Must successfully complete qualification(s).

🏝️ Benefits

• Compensation per approved task: a predetermined fixed rate for each task established in advance.

• Completely self-scheduled with no minimum hour requirements and no fixed timetable.

• Work remotely.

• No patient interaction and no malpractice risk.

• One-time onboarding bonus of $50, paid upon approval of the first task in the project.

• Quality incentives: additional earnings for high-quality approved tasks, as outlined in the project guidelines.

• Flexible, time-constrained project involvement.

• No time tracking or hour logging required.

• No tiers or speed adjustments affecting the rate.