Physician, Medical AI Evaluation

This is a fully remote position, open to applicants in Mexico, +5 more countries.

📋 Description

• Analyze clinical EHR vignettes along with corresponding questions, suggested answers, and misleading distractor options related to diagnosis and treatment tasks.

• Evaluate vignettes that cover the cardiovascular, nervous, hematologic, respiratory, digestive, urinary, reproductive, musculoskeletal, and integumentary systems.

• Assess the quality of clinical reasoning, the accuracy and completeness of vignettes, the content that reveals answers, the correctness and gradeability of answers, and the quality of traps.

• Confirm that reasoning chains derive answers solely from the facts presented in the vignettes.

• Rectify reasoning traces and craft concise rationales.

• Conduct three independent blind reviews followed by a physician adjudication process.

• Meet qualification requirements, engage in a project, complete assigned tasks, and submit work adhering to project timelines and acceptance criteria.

⛳️ Requirements

• Currently practicing physician with an MD or DO medical degree.

• Possess an active, unrestricted US medical license, verified during onboarding and re-checked on a monthly basis.

• Must reside outside the United States.

• Board certification or completion of residency training is required; attending-level candidates are preferred.

• Recent experience in direct patient care is essential.

• Extensive, multi-system diagnostic and treatment experience in internal, family, emergency, or hospital medicine is required.

• Strong clinical reasoning skills, encompassing differential diagnosis, next-step management, and adherence to evidence-based guidelines.

• Proficient in written English at a C1+ level.

• A CV submitted in English with an indication of English proficiency level is required.

• This position is not suitable for medical students, unlicensed medical graduates, non-physician clinicians, or individuals holding non-clinical healthcare titles.

🏝️ Benefits

• Compensation is provided per approved task at an equivalent rate of approximately $60/hr, with a single flat base rate for each approved task.

• Flexible contribution schedule and workload, estimated at 10–20 hours per week while the project is ongoing.

• Opportunity to contribute at any time and from any location.

• No patient interaction and no exposure to malpractice claims.

• A one-time onboarding reward of $50, paid upon approval of the first production task.

• Quality assurance and adjudication tasks are compensated at a higher rate.

• Additional earnings are available for high-quality approved tasks as part of the incentive program.