adversarial nli

**Adversarial NLI (ANLI)** is a **difficult NLI benchmark created through human-in-the-loop adversarial data collection** — examples that humans create specifically to fool state-of-the-art models, testing robust language understanding. **What Is Adversarial NLI?** - **Type**: Natural Language Inference benchmark. - **Method**: Humans write premise-hypothesis pairs that fool models. - **Difficulty**: Much harder than standard NLI datasets. - **Rounds**: Three rounds of increasingly difficult examples. - **Purpose**: Test model robustness and reasoning depth. **Why ANLI Matters** - **Robustness**: Exposes model weaknesses and shortcuts. - **Harder**: Models that ace SNLI/MultiNLI struggle on ANLI. - **Iterative**: Each round targets remaining model failures. - **Real Reasoning**: Requires genuine understanding, not shortcuts. - **Research Standard**: Used to evaluate robust NLU models. **Collection Process** 1. Human sees premise and model prediction. 2. Human writes hypothesis to fool the model. 3. If model fails, example added to dataset. 4. Repeat with improved models for harder rounds. **Performance Gap** - BERT on MultiNLI: ~86% - BERT on ANLI: ~45% (near random) ANLI is the **stress test for language understanding** — exposing shortcuts models learn.