searchqa
**SearchQA** is a question-answering dataset where answers must be found from multiple web search snippets, testing models' ability to aggregate evidence from noisy real-world sources.
## What Is SearchQA?
- **Size**: 140,000+ question-answer pairs
- **Source**: Jeopardy! questions with Google search snippets
- **Challenge**: Extract answers from 50+ noisy search results
- **Context**: Real web data, not curated paragraphs
## Why SearchQA Matters
Real-world QA involves searching the web, not reading a single clean document. SearchQA tests robustness to noise and evidence aggregation.
```
SearchQA Structure:
Question: "What is the capital of Australia?"
Search Snippets (noisy, redundant):
1. "...Sydney is the largest city in Australia..."
2. "...Canberra became the capital in 1913..."
3. "...Melbourne was briefly the capital..."
4. "...The Australian Parliament is in Canberra..."
...50+ snippets
Model must:
1. Filter irrelevant snippets
2. Aggregate evidence (Canberra appears multiple times)
3. Distinguish "largest" from "capital"
→ Answer: Canberra
```
**SearchQA Challenges**:
| Challenge | Description |
|-----------|-------------|
| Noise | Many snippets are irrelevant |
| Redundancy | Answer repeated differently |
| Distractors | Related but wrong entities |
| Length | 50+ documents to process |