minerva

**Minerva** is a **specialized mathematics language model developed by Google DeepMind by fine-tuning PaLM on 120B tokens of mathematical texts and competition problems**, engineering models specifically for mathematical reasoning and demonstrating that domain-focused training enables models to solve competition-grade math problems leveraging step-by-step chain-of-thought reasoning improved by compute-optimal fine-tuning. **Mathematics Specialization** Minerva proved that **mathematics requires different training**: | Training Data | Quantity | Source | |---|---|---| | University math textbooks | ~200GB | calculus, algebra, analysis | | Competition problems | ArithmeticComp, MATH dataset | AMC, AIME, IMO-level reasoning | | Academic papers | ArXiv mathematics sections | Proofs and formal reasoning | **Performance**: Minerva achieved **58.8% on MATH (competition-grade problems)** vs 50.3% for the base PaLM model—a dramatic improvement showing that domain specialization matters dramatically. **Chain-of-Thought Reasoning**: Minerva excels when models show step-by-step working—the reasoning ability compounds as models verbalize intermediate steps before providing final answers. **Limitations**: Minerva struggles with pure symbolic manipulation and sometimes hallucinates proofs—teaching researchers that LLMs capture reasoning patterns from data but cannot perform rigorous symbolic computation without external tools. **Legacy**: Established the template for specialized LLMs—fine-tune on domain-specific curated data, improve reasoning via step-by-step prompting, combine with external tools for unsolved problems. This approach influenced MathGPT, Llemma, and subsequent mathematics-specialized models.