metamath
**MetaMath** is a **mathematical reasoning model fine-tuned from Llama-2 using "In-Context Learning from Demonstrations" synthesized through prompt engineering, training on problem diversity rather than raw scale**, achieving competitive mathematical reasoning performance through synthetic data augmentation that teaches models to learn from diverse problem presentations rather than memorizing specific calculation patterns.
**Synthetic Data Strategy**
MetaMath pioneer the approach of **generating diverse mathematical representations**:
| Technique | Purpose | Outcome |
|-----------|---------|---------|
| **Problem Permutation** | Rephrase math problems in different ways | Models learn intent not surface patterns |
| **Step Variation** | Show same problem solved multiple ways | Captures reasoning flexibility |
| **Data Synthesis** | Generate synthetic math problems | Augment minority problem types |
Instead of collecting massive new datasets, MetaMath **augments existing data intelligently**, creating synthetic variations that expose models to problem diversity.
**Training Efficiency**: Achieves excellent performance with **moderate compute**—demonstrating that smart data (not just more data) improves mathematical reasoning.
**Performance**: Achieves **66.5% on GSM8K (grade school math)** and **18% on MATH (competition problems)**—competitive with much larger models through efficient training.
**Principled Approach**: Built on research into "in-context learning"—understanding how models learn from demonstrations vs memorization—enabling targeted training methodology.
**Legacy**: Established that **data quality and diversity outperform raw scale** in specialized domains—math reasoning improves more from 100K diverse problems than 1M repetitive calculations.