OpenRecently verified

Physician (MD/DO) - Medical AI Evaluation

  • Health & Medicine
  • Platform: Mindrift
  • Location: 36 countries

About this job

This project asks practicing physicians to review clinical case vignettes, each paired with a question, a proposed answer and a misleading distractor, to judge whether they are accurate, complete and fairly gradeable for an AI evaluation program. It suits physicians with broad multi-system experience in general and internal medicine who want part-time, project-based work. Each item goes through several independent blind reads followed by adjudication by a physician.

What you'll do

  • Evaluate clinical vignettes together with their questions, proposed answers and distractor answers.
  • Score benchmark items for vignette accuracy, completeness, answer correctness and trap quality.
  • Check that each reasoning chain reaches the answer from the vignette facts alone and write short rationales.
  • Complete three independent blind reads of each item before physician adjudication.

Requirements

Internal medicineClinical reasoningDifferential diagnosisEvidence-based guidelinesClinical vignette review
  • MD or DO

Pay

Pay not disclosed

The platform does not publish the pay for this job.

Location

Open to residents of: United Arab Emirates, Argentina, Bulgaria, Bahrain, Brazil, Canada, Chile, Colombia, Czechia, Egypt, United Kingdom, Hungary, Indonesia, Israel, India, Jamaica, Jordan, Japan, South Korea, Kuwait, Lebanon, Mexico, Malaysia, New Zealand, Oman, Peru, Philippines, Poland, Qatar, Romania, Saudi Arabia, Singapore, Slovakia, Türkiye, United States, South Africa.

How to apply

You apply on Mindrift, the AI training platform run by Toloka: submit your application for the role, pass a qualification step, then work on projects that match your expertise.

All Mindrift jobs and how the platform works

Source

Official posting: https://apply.workable.com/j/515987B230

Last checked on October 10, 2026.

Similar jobs

  • Health & Medicine
  • Platform: Mindrift
  • Location: United States

This project-based role asks practicing physicians to review clinical case vignettes used to test medical AI systems. The work centers on general and internal medicine, covering diagnosis and treatment questions across many organ systems. It suits licensed doctors who can give part-time hours to independent review and written rationales.

Posted October 9, 2026

OpenRecently verifiedPay not disclosed
  • Health & Medicine
  • Platform: Mindrift
  • Location: United States

This project asks practicing physicians to review clinical case vignettes, with their questions and answer options, for a medical AI evaluation program. The work centers on general and internal medicine and checks whether each item is accurate, gradeable, and free of giveaways. It suits clinicians who can take on part-time, project-based assignments.

Posted October 9, 2026

OpenRecently verifiedPay not disclosed
  • Health & Medicine
  • Platform: Mercor
  • Location: United States

This role asks practicing physicians to judge how well health AI models handle diagnoses, clinical reasoning and care plans. The work is part-time and asynchronous, aimed at MDs and DOs who currently see patients in primary care, internal medicine or hospital-based specialties.

Posted October 8, 2026

OpenRecently verified$150/hr

Referral link: we may earn a fee. Apply without it

  • Health & Medicine
  • Platform: Terac
  • Location: United States

This remote project asks practicing hospitalists and family medicine physicians to judge how accurately clinical AI tools suggest diagnoses from patient charts. Participants read de-identified records, reach their own diagnosis, and then decide whether an AI model's proposed diagnoses are correct and clinically relevant. It suits licensed physicians with strong differential diagnosis skills who can commit at least five hours within a set window.

Posted October 9, 2026

  • Health & Medicine
  • Platform: micro1
  • Location: 58 countries

This part-time contract asks physicians to review clinical cases, patient data, and AI-generated medical content, giving expert feedback that helps train AI systems supporting medical decisions. It is aimed at practicing doctors, especially those from general specialties such as internal medicine, family practice, or emergency medicine. No prior AI experience is needed, since clinical knowledge is the main requirement.

Posted October 6, 2026

OpenRecently verifiedPay not disclosed

Referral link to micro1's job list: search for this role there. Open this exact job

  • Health & Medicine
  • Platform: Mercor
  • Location: United States
  • Level: Intermediate

Mercor is recruiting experienced nurse practitioners to evaluate and improve cutting-edge AI systems in healthcare. You will analyze clinical content generated by AI, treatment plans and patient education materials by leveraging your real-world expertise in diagnosing, treating and managing patients with acute and chronic conditions.

Posted August 21, 2026

OpenRecently verified$120-150/hr

Referral link: we may earn a fee. Apply without it

Keep exploring