GSM8K is a benchmark dataset of 8,500 linguistically diverse grade school math word problems designed to evaluate mathematical reasoning capabilities in language models.
What Is GSM8K?
- Size: 7,500 training + 1,000 test problems
- Difficulty: Grade school level (ages 6-12)
- Format: Word problems requiring 2-8 step solutions
- Metric: Exact match accuracy on final numerical answer
Why GSM8K Matters
Math reasoning is a key capability for AI assistants. GSM8K tests multi-step deductive reasoning that requires understanding language and applying arithmetic.
GSM8K Example Problem:
"Janet has 3 apples. She buys 2 more apples, then
gives half of all her apples to her friend. How
many apples does Janet have?"
Solution steps:
1. Start: 3 apples
2. Buy more: 3 + 2 = 5 apples
3. Give half: 5 / 2 = 2.5 → 2 apples (integer)
Answer: 2
Model Performance (2024):
| Model | GSM8K Accuracy |
|---|---|
| GPT-4 | ~92% |
| Claude 3 | ~88% |
| Llama 2 70B | ~57% |
| Chain-of-thought prompting | +10-20% improvement |
gsm8kmath benchmarkword problems
Explore 500+ Semiconductor & AI Topics
From EUV lithography to CUDA optimization — search the full knowledge base or chat with our AI assistant.