MathQA is the large-scale math word problem dataset annotated with executable operation programs — bridging the gap between end-to-end answer prediction and interpretable program synthesis by requiring models to produce a structured formula tree that explicitly encodes the mathematical operations needed to solve each problem.

What Is MathQA?

• Scale: ~37,200 problems from AQuA-RAT, re-annotated with operation programs.
• Format: Multiple-choice question + natural language rationale + structured operation program.
• Operation Language: A domain-specific functional language: divide(n1, n2), multiply(n1, n2), add(n1, subtract(n2, n3)) — composable arithmetic operations over extracted numbers.
• Subjects: Algebra, Arithmetic, Probability, Geometry, Physics, and General word problems.
• Goal: Map natural language problem text to an executable program that produces the correct answer.

The Three-Part Annotation

Each MathQA example contains:

1. Problem Text: "A train travels from city A to city B at 60 mph. The return trip is at 40 mph. What is the average speed for the entire trip?"

2. Rationale (Natural Language): "Average speed = total distance / total time. Let d be the one-way distance. Time AB = d/60, time BA = d/40, total time = d/60 + d/40 = 5d/120. Average = 2d / (5d/120) = 48 mph."

3. Operation Program: divide(multiply(2, 60), add(divide(60, 40), divide(40, 60))) (simplified symbolic form)

Why Operation Programs Matter

Standard seq2seq math solvers (directly predicting the answer number) have three critical weaknesses:

• Unverifiable: A correct answer could come from wrong reasoning — no way to audit intermediate steps.
• Non-compositional: Cannot generalize to problems requiring a new combination of operations.
• Brittle: Small perturbations cause catastrophic failures because there's no structured representation to fall back on.

Operation programs address all three:

• Auditable: Every step of the computation is explicit and inspectable.
• Compositional: New problems can be solved by recombining known operations.
• Executable: The program can be run against a symbolic interpreter to verify correctness independently of the neural model.

Why MathQA Matters

• Toward Neural Program Synthesis: MathQA positioned math reasoning as a program synthesis problem, connecting NLP to the formal methods community.
• Intermediate Representation: Inspired later work on tool-augmented LLMs (code generation for math), where models write Python code rather than predict answers.
• Few-Shot Curriculum: The annotated rationales became a template for Chain-of-Thought fine-tuning.
• Baseline Difficulty: Even with structured program targets, seq2seq models achieve only ~70-75% accuracy — substantial error in a domain where the answer is always verifiable.
• Dataset Noise Warning: MathQA has known annotation inconsistencies — some operation programs do not match the natural language rationale. Researchers should use with caution and cross-reference.

Performance Benchmarks

Connection to Downstream Work

MathQA directly influenced:

• PoT (Program-of-Thought): Generate Python code for math problems, execute for the answer.
• PAL (Program-Aided Language Models): Use LLMs as code generators, Python interpreter as the solver.
• Tool-Use Agents: LLMs calling external calculators (Wolfram Alpha, sympy) for reliable numeric computation.

MathQA is showing your mathematical work in executable form — requiring the model to produce not just the answer but the precise sequence of operations that derives it, making math reasoning transparent, auditable, and composable.