Physician Reviewer, Medical AI Evaluation

This is a fully remote position, open to applicants in Japan, +3 more countries.

📋 Description

• Assess clinical EHR vignettes alongside questions, suggested answers, and misleading “trap” answers related to diagnosis and treatment tasks.

• Evaluate the quality of clinical reasoning, the accuracy and completeness of vignettes, the correctness and gradeability of answers, and the quality of traps.

• Determine if reasoning chains derive answers solely from the facts presented in the vignettes.

• Correct reasoning paths and compose brief rationales.

• Engage in three independent blind evaluations followed by physician adjudication.

• Fulfill project responsibilities within the designated timeline and established acceptance criteria.

• Complete qualifications, join a project, execute tasks, and receive compensation.

⛳️ Requirements

• Actively practicing physician residing outside the US.

• Medical degree (MD or DO).

• Active, unrestricted US medical license, verified during onboarding and re-checked on a monthly basis.

• Board certification or completion of residency training.

• Recent direct patient care experience; attending-level preferred.

• Extensive, multi-system diagnostic and treatment experience in internal, family, emergency, or hospital medicine.

• Strong clinical reasoning skills, including differential diagnosis, next steps in management, and the application of evidence-based guidelines.

• Previous experience in medical AI evaluation, clinical content or exam question review, or medical annotation/QA is advantageous.

• Proficient written English (C1+).

• CV submitted in English.

• Not suitable for medical students, unlicensed medical graduates, non-physician clinicians, or individuals in non-clinical healthcare roles.

🏝️ Benefits

• Approximately 10–20 hours per week available while the project is ongoing; flexibility to choose when and how much to contribute.

• Ability to contribute at any time, from any location.

• No patient interaction required.

• No malpractice liability exposure.

• $50 one-time onboarding reward, issued upon approval of your first production task.

• Incentives: additional income for high-quality tasks that are approved.

• QA and adjudication tasks are compensated at a higher rate.