LLM Evaluation and Benchmarking is the systematic methodology for measuring language model capabilities across diverse tasks — encompassing academic benchmarks (MMLU, HumanEval, GSM8K), arena-style human evaluation (Chatbot Arena), and automated frameworks (lm-evaluation-harness, OpenCompass), where the design of evaluation protocols, metric selection, and contamination prevention are critical challenges that determine whether benchmark scores reflect genuine capability or test-set overfitting.

Evaluation Taxonomy

Key Benchmarks

Benchmark Contamination

Problem: Benchmark questions appear in training data
         → Model memorizes answers, scores inflate

Contamination vectors:
  - Direct: Benchmark hosted on GitHub → crawled into training data
  - Indirect: Benchmark discussed in blogs/forums → answers in training data
  - Paraphrased: Slight rephrasing still triggers memorization

Detection methods:
  - n-gram overlap between training data and benchmark
  - Canary strings: Insert unique markers, check if model reproduces
  - Performance on rephrased vs. original questions

LLM-as-Judge

# Using GPT-4 as automated evaluator
prompt = f"""Rate the quality of this response on a scale of 1-10.
Question: {question}
Response A: {response_a}
Response B: {response_b}
Which is better and why?"""
# Issues: Position bias (prefers first), verbosity bias, self-preference
# Mitigation: Swap positions, average scores, use multiple judges

Chatbot Arena (LMSYS)

• Users submit questions → two anonymous models respond → user picks winner.
• Elo rating system ranks models.
• 1M+ human votes → statistically robust.
• Best holistic measure of "real-world" LLM quality.
• Weakness: Biased toward chat/creative tasks, less rigorous on technical.

Evaluation Frameworks

LLM evaluation is the unsolved meta-problem of AI development — while individual benchmarks measure specific capabilities, no single evaluation captures the full range of model quality, and the field struggles with benchmark saturation, contamination, and the tension between reproducible automated metrics and holistic human assessment, making evaluation methodology itself one of the most active and important research areas in AI.