med palm
**Med-PaLM** is a **medical domain language model developed by Google Research that was the first AI system to achieve "expert" level performance on the US Medical Licensing Exam (USMLE)** — with Med-PaLM 2 reaching 86.5% accuracy on MedQA (surpassing the ~60% passing threshold by a wide margin), built by fine-tuning Google's PaLM foundation model using instruction tuning on curated medical question-answering datasets (MultiMedQA) and rigorously evaluated for clinical safety, accuracy, and potential harm.
**What Is Med-PaLM?**
- **Definition**: A series of medical AI models built on top of Google's PaLM (Pathways Language Model) — fine-tuned with medical instruction data and evaluated against clinical expert benchmarks, designed for medical question answering, clinical reasoning, and health information retrieval.
- **Med-PaLM 1**: First AI to pass the USMLE with ~67% accuracy — demonstrating that LLMs fine-tuned on medical data could match the knowledge threshold required of human physicians.
- **Med-PaLM 2**: Dramatically improved to 86.5% accuracy on MedQA — reaching "expert physician" level performance and significantly outperforming both ChatGPT (~60%) and the original Med-PaLM on all medical benchmarks.
- **MultiMedQA**: Google's comprehensive medical evaluation benchmark combining MedQA (USMLE questions), MedMCQA (Indian medical entrance exams), PubMedQA (biomedical literature questions), and additional clinical QA datasets.
**Performance Evolution**
| Model | MedQA (USMLE) | MedMCQA | PubMedQA | Notes |
|-------|-------------- |---------|----------|-------|
| Med-PaLM 1 (2022) | 67.6% | 57.6% | 79.0% | First AI to pass USMLE |
| Med-PaLM 2 (2023) | 86.5% | 72.3% | 81.8% | Expert physician level |
| GPT-4 (2023) | ~86% | ~70% | ~80% | Comparable to Med-PaLM 2 |
| ChatGPT (GPT-3.5) | ~60% | ~55% | ~75% | Near passing threshold |
**Safety and Evaluation**
- **Clinical Expert Review**: Med-PaLM responses evaluated by panels of practicing physicians across 9 dimensions — scientific accuracy, potential harm, evidence of reasoning, demographic bias, likelihood of clinical action disagreement.
- **Harm Assessment**: Every response evaluated for potential patient harm — critical for medical AI where incorrect advice could lead to delayed treatment, wrong medication, or missed diagnoses.
- **Physician Comparison**: In blind evaluations, Med-PaLM 2 answers were preferred over physician answers 40% of the time — showing AI can match or exceed human quality on structured medical questions.
- **Limitations Acknowledged**: Google explicitly documents that Med-PaLM is not approved for clinical use — it's a research demonstration, not a diagnostic tool, and real clinical deployment requires regulatory approval (FDA, CE marking).
**Deployment and Access**
- **Pilot Program**: Google piloting Med-PaLM with select healthcare partners (Mayo Clinic, HCA Healthcare) for clinical decision support — not direct patient interaction.
- **Google Cloud integration**: Available through Vertex AI for approved healthcare research institutions.
- **Not publicly available**: Unlike open-source alternatives (Meditron), Med-PaLM remains closed-access due to safety concerns around uncontrolled medical AI deployment.
**Med-PaLM is the benchmark-setting medical AI that proved language models can reach physician-level accuracy on structured medical examinations** — while simultaneously demonstrating the critical importance of rigorous safety evaluation, harm assessment, and controlled deployment for AI systems operating in high-stakes clinical domains.