Physician Reviewer, MD/DO – Medical AI Evaluation

This is a fully remote position, open to applicants in United States.

📋 Description

• Assess clinical EHR vignettes alongside questions, suggested answers, and distractor trap answers related to diagnosis and treatment tasks.

• Examine cases across cardiovascular, nervous, hematologic, respiratory, digestive, urinary, reproductive, musculoskeletal, and integumentary systems.

• Evaluate the quality of clinical reasoning, accuracy and completeness of vignettes, correctness and gradeability of answers, and quality of distractors.

• Determine if reasoning chains arrive at conclusions based solely on the facts presented in the vignettes.

• Correct reasoning pathways and compose brief rationales.

• Engage in three independent blind reads, followed by adjudication by physicians.

• Test, assess, and enhance medical AI systems for leading technology companies.

⛳️ Requirements

• Must be a US-based, actively practicing physician (MD/DO).

• Possess a medical degree (MD or DO).

• Hold an active, unrestricted US medical license, verified with the relevant medical board.

• Have board certification or completed residency training.

• Recent direct patient care experience is required; attending-level is preferred.

• Extensive diagnostic and treatment experience across multiple systems in internal, family, emergency, or hospital medicine.

• Demonstrate strong clinical reasoning skills, including differential diagnosis, next-step management, and adherence to evidence-based guidelines.

• Prior experience in evaluating medical AI, reviewing clinical content or exam questions, or conducting medical annotation/QA is advantageous.

• Proficient written English skills (C1+ level).

• Not appropriate for medical students, unlicensed medical graduates, non-physician clinicians, or individuals with non-clinical healthcare titles.

• Availability to work approximately 10–20 hours per week during active project phases.

• Ability to meet deadlines for task submissions and comply with specified acceptance criteria.

🏝️ Benefits

• Participation is project-based.

• Opportunities for part-time project work.

• Compensation is provided per accepted task.

• Earnings can reach up to the equivalent of $100/hr, depending on qualification level and task efficiency.