ms marco

**MS MARCO (Microsoft MAchine Reading COmprehension)** is a **massive-scale dataset for Reading Comprehension and Passage Ranking, derived from real Bing search queries** — containing 1M+ queries and partially human-generated answers, it is the standard benchmark for Neural Information Retrieval (IR). **Tasks** - **Passage Ranking**: Given a query, rank 1000 passages by relevance. (The "TREC" of the Deep Learning era). - **Answer Generation**: Generate a natural language answer based on the retrieved passages. - **Key**: Many queries have "No Answer" in the top passages. **Why It Matters** - **Scale**: Large enough to train data-hungry Transformers from scratch. - **Retrieval**: The definitive benchmark for Dense Retrieval (DPR) and Re-ranking models (Cross-Encoders). - **Realism**: Queries are short, noisy, and real ("how to cook pasta", "social security office hours"). **MS MARCO** is **the search engine test** — the definitive benchmark for teaching AI how to retrieve and rank relevant information from the web.