Benchmark Contamination
Benchmark contamination is the situation in which a model's training data contains some or all of a benchmark's test questions, answers, or close paraphrases, so that a high score may reflect memorization rather than the capability the benchmark was designed to measure. It is a special case of train-test leakage, an old problem in machine learning, but it is especially acute for large language models because their training corpora are scraped from the web, where benchmarks, solutions, and discussions of them are posted publicly.
The mechanism is mundane: a benchmark is published online, copied into repositories, blog posts, forums, and study sites, and then swept up by a web-crawl such as Common Crawl that feeds pretraining. No one needs to cheat deliberately. The leak can be direct (the test set itself), indirect (a dataset derived from the same source), or subtle (answers appearing without the questions, or the test set used for tuning a prompt or training recipe). Fine-tuning on data that overlaps a benchmark, or selecting model checkpoints by test score, produces the same inflation by a different route. Contamination becomes more likely the older and more popular a benchmark is.
The standard detection method is n-gram overlap: flag a test example if a sequence of n consecutive words or tokens from it appears in the training corpus. The GPT-3 paper (Brown et al., OpenAI, 2020) did this systematically, looking for overlaps of about 13 words between benchmark examples and the training set, and then reported results separately on the "clean" subset of each benchmark and compared them with the full set. The authors also disclosed that a bug in their filtering step meant some overlaps were not removed, and that retraining was too costly, so they analyzed the effect instead and found that for most benchmarks the contaminated portion made little difference, with a few flagged exceptions. This candid analysis became a template, and later technical reports from major labs include similar contamination sections.
N-gram checks are necessary but weak, because paraphrase, translation, or reformatting defeats exact matching. A question rewritten in different words, or the same problem in another language, can be memorized without a single long n-gram match. Researchers have therefore proposed embedding-based similarity, asking a model to complete a test item verbatim from a partial prompt, testing whether a model prefers the original ordering of items over shuffled ones, and comparing performance on a benchmark against fresh, similar problems. Zhang et al. (2024) built GSM1k, a new set of grade-school math problems matched to GSM8K in style, and reported that several model families scored notably lower on it than on the original, which they read as evidence of overfitting to the public set.
Dynamic or time-split benchmarks attack the problem structurally rather than by detection. Dynabench (Kiela et al., 2021) used human-in-the-loop adversarial collection, and later efforts such as LiveCodeBench and LiveBench (both 2024) continually add new questions drawn from recent contests or sources published after model training cutoffs, so that a model could not have seen them. Private held-out test sets, where the evaluator never publishes the answers, serve a similar role but require trusting the evaluator. Each approach has a cost: dynamic benchmarks make results across dates harder to compare, and private ones reduce transparency.
The common misconception is that contamination is a binary accusation of cheating. In practice it is a spectrum of exposure with a measurable and often modest effect, and evidence of exposure is not proof that the model relies on it. The opposite error, assuming a high score is clean because no overlap was found, is just as common. Careful reporting states the detection method used, the n chosen, the share of items flagged, and the score with and without them, and treats leaderboard numbers on old, famous benchmarks with appropriate caution.
Read benchmark contamination as the exam question that leaked into the textbook: the test is only as informative as the model's inability to have seen it, so every score deserves a note on how that was checked.