glue benchmark

**GLUE (General Language Understanding Evaluation)** is a **collection of 9 diverse NLU tasks (QA, NLI, Sentiment, Paraphrasing) combined into a single benchmark metric** — introduced in 2018, it standardized model evaluation and drove the "pre-train then fine-tune" revolution (BERT era). **Tasks** - **MNLI/RTE**: Inference. - **QQP/MRPC**: Paraphrase/Similarity. - **SST-2**: Sentiment. - **CoLA**: Linguistic Acceptability (Grammar). - **STS-B**: Semantic Similarity. - **QNLI**: QA-NLI. - **WNLI**: Winograd (often excluded due to issues). **Why It Matters** - **Standardization**: Before GLUE, everyone purely tested on ImageNet or custom splits. GLUE created a shared leaderboard. - **Solved**: BERT and RoBERTa quickly saturated GLUE (surpassed human baseline), necessitating SuperGLUE. - **Generalization**: Forced models to be "generalists" (one model, many tasks). **GLUE Benchmark** is **the SAT for AI** — the first standardized test suite that measured general language understanding capabilities across multiple domains.