medmcqa
**MedMCQA** is the **large-scale Indian medical entrance exam benchmark** — containing 194,000 multiple-choice questions from AIIMS (All India Institute of Medical Sciences) and NEET-PG (National Eligibility Entrance Test for Postgraduate Medicine) examinations, providing the largest publicly available medical MCQ dataset for training and evaluating AI clinical reasoning systems across the full spectrum of medical knowledge.
**What Is MedMCQA?**
- **Origin**: Pal et al. (2022).
- **Scale**: 194,000 questions — the largest public medical MCQ dataset.
- **Source**: AIIMS and NEET-PG entrance examinations (2000-2021).
- **Format**: 4-choice MCQ with explanations for ~25% of questions.
- **Subjects**: 21 medical subjects covering all clinical and basic science disciplines.
- **Splits**: 182,822 training, 4,183 validation, 6,150 test.
**The 21 Medical Subjects**
Basic Sciences: Anatomy, Physiology, Biochemistry, Pathology, Pharmacology, Microbiology, Forensic Medicine
Clinical Sciences: Medicine, Surgery, Pediatrics, Obstetrics & Gynecology, Ophthalmology, ENT, Psychiatry, Dermatology, Anesthesia, Radiology, Orthopedics, Community Medicine, Dental
**Why MedMCQA Complements USMLE-Based Benchmarks**
MedMCQA reflects the Indian medical education system, which differs from USMLE in important ways:
- **Drug Formulary**: Questions reference drugs approved in India, including older antibiotics and antiparasitics common in tropical medicine but rare in USMLE.
- **Disease Prevalence**: Malaria, tuberculosis, leprosy, and dengue appear frequently — reflecting Indian epidemiology. USMLE rarely tests these.
- **Traditional Question Style**: AIIMS questions are known for testing highly specific anatomical facts and pharmacological details that require precise memorization.
- **Explanations Available**: ~25% of MedMCQA examples include expert explanations — valuable for Chain-of-Thought supervised learning.
**Performance Results**
| Model | MedMCQA Accuracy |
|-------|----------------|
| Random baseline | 25.0% |
| AIIMS passing threshold (human) | ~60% |
| BERT fine-tuned | 53.2% |
| PubMedBERT fine-tuned | 57.1% |
| GPT-3.5 | 61.3% |
| GPT-4 | 79.1% |
| Med-PaLM 2 | 75.2% |
**Why MedMCQA Matters**
- **Global Medical AI Coverage**: US-centric benchmarks miss tropical medicine, nutrition-related diseases, and Global South epidemiology. MedMCQA ensures AI medical tools work beyond North America.
- **Scale for Pretraining**: 182,000 training questions is large enough for specialized fine-tuning — enabling medical LLMs trained on MedMCQA to demonstrate measurably improved clinical knowledge.
- **Explanation-Based Learning**: The subset with explanations enables process supervision training — teaching models to reason through clinical questions step-by-step.
- **Indian Healthcare AI Market**: With 1.4 billion people and a shortage of physicians in rural areas, AI clinical decision support trained to NEET-PG standards has direct deployment potential.
- **Benchmark Diversity**: A comprehensive medical AI evaluation framework must include MedMCQA alongside MedQA (USMLE) and PubMedQA — single-exam evaluation misses domain breadth.
MedMCQA is **the medical entrance exam at scale** — providing 194,000 questions from India's most competitive medical examinations to train and evaluate clinical AI systems, ensuring that medical AI competence is measured across global medical education systems rather than only the US USMLE standard.