meditron
**Meditron** is a **suite of open-source medical language models (7B and 70B parameters) developed by EPFL (Swiss Federal Institute of Technology) based on Llama 2, achieving state-of-the-art performance on medical question-answering benchmarks among open-source models** — using a novel GAP-Replay continual learning technique to train on PubMed articles, medical guidelines, and clinical textbooks without catastrophically forgetting the general English knowledge required for coherent medical conversations.
**What Is Meditron?**
- **Definition**: Open-source medical LLMs fine-tuned from Llama 2 on curated medical corpora — designed for clinical decision support, medical education, and diagnostic assistance, with particular focus on accessibility for low-resource healthcare settings where commercial AI APIs are prohibitively expensive.
- **GAP-Replay**: A continual learning method that replays a small percentage of general-purpose training data alongside medical data during fine-tuning — preventing "catastrophic forgetting" where the model loses ability to hold coherent conversations while gaining medical knowledge.
- **Medical Data Sources**: PubMed (biomedical literature), clinical practice guidelines (WHO, NIH), medical textbooks, and curated Q&A datasets — carefully filtered for accuracy and relevance.
- **Low-Resource Design**: The 7B model runs on a single consumer GPU (16 GB VRAM) — enabling hospitals in developing nations to deploy local medical AI assistants without sending private patient data to cloud APIs.
**Performance on Medical Benchmarks**
| Benchmark | Meditron-70B | Llama-2-70B (base) | GPT-3.5 | Med-PaLM 2 |
|-----------|-------------|-------------------|---------|------------|
| MedQA (USMLE) | 70.2% | 55.1% | 60.2% | 86.5% |
| MedMCQA (Indian medical) | 62.3% | 48.7% | 55.8% | 72.3% |
| PubMedQA | 81.6% | 73.2% | 75.1% | 81.8% |
**Key Features**
- **Clinical Reasoning**: Meditron generates step-by-step diagnostic reasoning — presenting differential diagnoses with supporting evidence from medical literature, not just single-word answers.
- **Multilingual Medical**: Built on Llama 2's multilingual foundation, enabling medical assistance in languages underserved by English-centric medical AI.
- **Safety Design**: Trained with conservative refusal patterns for high-risk scenarios — directing patients to seek professional medical care rather than providing definitive diagnoses.
- **Reproducible Research**: Full training code, data processing pipeline, and evaluation scripts publicly available — enabling the medical AI research community to build upon and improve the methodology.
**Meditron vs. Other Medical LLMs**
| Model | Organization | Access | Size | MedQA Score |
|-------|-------------|--------|------|------------|
| **Meditron** | EPFL | Open-source | 7B, 70B | 70.2% |
| Med-PaLM 2 | Google | Closed API | Unknown | 86.5% |
| BioMistral | Open | Open-source | 7B | 58.9% |
| PMC-LLaMA | Open | Open-source | 13B | 62.1% |
| ClinicalGPT | Closed | Closed | Unknown | ~60% |
**Meditron is the leading open-source medical language model** — demonstrating that domain-specialized fine-tuning with continual learning techniques can produce clinically useful AI assistants accessible to healthcare institutions worldwide, including resource-limited settings where data privacy and cost make commercial APIs infeasible.