OpenRecently verified

Physician Reviewer (MD/DO) - Medical AI Evaluation

  • Health & Medicine
  • Platform: Mindrift
  • Location: 36 countries

About this job

This project asks practicing physicians to review clinical EHR vignettes written as benchmark items for AI evaluation, checking accuracy, answer correctness and the quality of distractor options. It is aimed at clinicians with broad multi-system experience in general and internal medicine who can take on part-time, project-based assignments.

What you'll do

  • Assess clinical EHR vignettes, questions, proposed answers and distractor options across multiple body systems.
  • Score the clinical reasoning quality and gradeability of benchmark items.
  • Check that each reasoning chain reaches the answer from vignette facts alone and write short rationales.
  • Complete independent blind reads before physician adjudication.

Requirements

Internal medicineClinical reasoningDifferential diagnosisEvidence-based guidelinesElectronic health recordsFamily medicineEmergency medicineHospital medicine
  • MD or DO

Pay

Pay not disclosed

The platform does not publish the pay for this job.

Location

Open to residents of: United Arab Emirates, Argentina, Bulgaria, Bahrain, Brazil, Canada, Chile, Colombia, Czechia, Egypt, United Kingdom, Hungary, Indonesia, Israel, India, Jamaica, Jordan, Japan, South Korea, Kuwait, Lebanon, Mexico, Malaysia, New Zealand, Oman, Peru, Philippines, Poland, Qatar, Romania, Saudi Arabia, Singapore, Slovakia, Türkiye, United States, South Africa.

How to apply

You apply on Mindrift, the AI training platform run by Toloka: submit your application for the role, pass a qualification step, then work on projects that match your expertise.

All Mindrift jobs and how the platform works

Source

Official posting: https://apply.workable.com/j/2A873BAC39

Last checked on October 17, 2026.

Similar jobs

  • Health & Medicine
  • Platform: micro1
  • Location: 58 countries
  • Language: Korean

This part-time remote role asks practising physicians to review and annotate clinical cases and AI-generated medical content so that models reason more accurately about medicine. It suits Korean-speaking doctors with clinical experience, ideally in general specialties, and no prior AI background is needed.

Posted October 14, 2026

OpenRecently verifiedPay not disclosed

Referral link to micro1's job list: search for this role there. Open this exact job

  • Health & Medicine
  • Platform: Mindrift
  • Location: United States

This project asks practicing physicians to review clinical case vignettes, with their questions and answer options, for a medical AI evaluation program. The work centers on general and internal medicine and checks whether each item is accurate, gradeable, and free of giveaways. It suits clinicians who can take on part-time, project-based assignments.

Posted October 9, 2026

OpenRecently verifiedPay not disclosed
  • Health & Medicine
  • Platform: micro1
  • Location: 58 countries
  • Language: Japanese

This contract role asks practising physicians to review and annotate clinical cases and medical content so that AI systems give medically accurate answers. It suits doctors with recent clinical experience who are fluent in Japanese and English. The work is part-time and fully remote.

Posted October 14, 2026

OpenRecently verifiedPay not disclosed

Referral link to micro1's job list: search for this role there. Open this exact job

  • Health & Medicine
  • Platform: Mindrift
  • Location: United States

This project asks practicing physicians to review clinical case vignettes written for evaluating medical AI systems. The work involves judging whether each vignette, question, correct answer and decoy answer are accurate, complete and fair, and writing short rationales. It suits board-certified or residency-trained clinicians with recent patient care and broad multi-system experience.

Posted October 13, 2026

OpenRecently verifiedPay not disclosed
  • Health & Medicine
  • Platform: Mercor
  • Location: United States

This role asks practicing physicians to judge how well health AI models handle diagnoses, clinical reasoning and care plans. The work is part-time and asynchronous, aimed at MDs and DOs who currently see patients in primary care, internal medicine or hospital-based specialties.

Posted October 8, 2026

OpenRecently verified$150/hr

Referral link: we may earn a fee. Apply without it

  • Health & Medicine
  • Platform: Mercor
  • Location: United States

This part-time remote role asks a physician with a pediatric emergency or urgent care background to review clinical documentation and patient charts for accuracy, completeness and clinical soundness. The work also involves checking AI-generated clinical summaries and giving structured written feedback that product teams can act on. It suits clinicians who want to keep an active practice while helping shape medical AI tools.

Posted October 14, 2026

OpenRecently verified$150-170/hr

Referral link: we may earn a fee. Apply without it

Keep exploring