AQuA-RAT (Algebra Question Answering with Rationales) is the 100,000-question algebra dataset where every problem comes with a human-written natural language rationale explaining the solution step-by-step — one of the foundational datasets that demonstrated how explicit reasoning steps improve both model training and interpretability, directly inspiring the Chain-of-Thought prompting paradigm.

What Is AQuA-RAT?

• Scale: ~100,000 algebra and arithmetic problems (large for its era).
• Format: Multiple-choice (5 options: A/B/C/D/E) + free-form natural language rationale.
• Source: Problems crowdsourced via Amazon Mechanical Turk and adapted from GRE/GMAT preparation materials.
• Coverage: Ratio and proportion, percent, average, speed/distance/time, profit and loss, linear equations, simple probability.
• Rationale Format: "First, let x = the original price. Then 0.8x = 40, so x = 50. The answer is C."

The Rationale Innovation

Before AQuA-RAT, math datasets provided only (problem, answer) pairs. AQuA-RAT added the critical third element: the reasoning chain. This enables:

• Process Supervision: Train models on correct intermediate steps, not just final answers.
• Error Attribution: When a model is wrong, examine the rationale to find where reasoning broke down.
• CoT Template Generation: AQuA-RAT rationales served as templates for manually crafting Chain-of-Thought few-shot examples in Wei et al. (2022), the seminal CoT paper.
• Student Modeling: Educational AI can compare a student's reasoning chain to the gold rationale to identify misconceptions.

Connection to Chain-of-Thought

The 2022 paper "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models" used AQuA-RAT as one of its five benchmark tasks. The key insight — that providing step-by-step reasoning examples in the prompt dramatically improved LLM performance on math problems — was demonstrated on AQuA-RAT alongside GSM8K, SVAMP, MAWPS, and MATH.

Why AQuA-RAT Matters

• Historical Significance: One of the first large-scale datasets with natural language reasoning annotations for math — pioneered the idea that explanations improve AI math performance.
• GRE/GMAT Difficulty: Problems are at the standardized test level, requiring algebraic setup (not just arithmetic). This is harder than primary-school word problems (MAWPS) but accessible without competition-level insight (MATH).
• Multi-Step Reasoning: Most problems require 3-5 logical steps, making them ideal for CoT evaluation.
• Curriculum Learning: The rationale quality varies (crowdsourced annotation has noise), making AQuA-RAT useful for studying how model performance degrades with noisy supervision.
• Broad Coverage: GRE/GMAT topics are directly relevant to standardized test preparation AI and educational technology.

Known Limitations

• Annotation Noise: Some crowdsourced rationales contain arithmetic errors or unclear steps (~5-10% estimated noisy examples).
• Limited Symbolic Diversity: Compared to MATH (competition level), AQuA-RAT problems are formulaic — the same problem structures repeat with different numbers.
• English Only: No multilingual variants, limiting use in international educational AI research.

Datasets It Inspired

• MathQA: Re-annotated AQuA-RAT problems with structured operation programs.
• GSM8K: More carefully crowdsourced grade-school math with clean step-by-step rationales.
• ORCA: Used AQuA-RAT-style rationale generation at scale with LLM-generated explanations.

AQuA-RAT is the algebra textbook that taught AI to show its work — proving that natural language reasoning chains are not just interpretability aids but genuine performance boosters, laying the intellectual foundation for the Chain-of-Thought era of language model development.