Minerva is a specialized mathematics language model developed by Google DeepMind by fine-tuning PaLM on 120B tokens of mathematical texts and competition problems, engineering models specifically for mathematical reasoning and demonstrating that domain-focused training enables models to solve competition-grade math problems leveraging step-by-step chain-of-thought reasoning improved by compute-optimal fine-tuning.

Mathematics Specialization

Minerva proved that mathematics requires different training:

Performance: Minerva achieved 58.8% on MATH (competition-grade problems) vs 50.3% for the base PaLM model—a dramatic improvement showing that domain specialization matters dramatically.

Chain-of-Thought Reasoning: Minerva excels when models show step-by-step working—the reasoning ability compounds as models verbalize intermediate steps before providing final answers.

Limitations: Minerva struggles with pure symbolic manipulation and sometimes hallucinates proofs—teaching researchers that LLMs capture reasoning patterns from data but cannot perform rigorous symbolic computation without external tools.

Legacy: Established the template for specialized LLMs—fine-tune on domain-specific curated data, improve reasoning via step-by-step prompting, combine with external tools for unsolved problems. This approach influenced MathGPT, Llemma, and subsequent mathematics-specialized models.