Question Answering Machine Reading Comprehension
# Question Answering: Machine Reading & Comprehension
## Introduction & Motivation
Question Answering: answer questions about passages. Machine Reading Comprehension: extract answers from text. Span prediction tasks. Applications: information retrieval, search, assistants.
Motivation: Understand questions and documents; extract relevant information.
Applications: Search, assistants, information access.
---
## Core Concepts & Theory
### Passage Encoding
Represent context document.
### Question Encoding
Represent query document.
### Answer Prediction
Span or free-form generation.
---
## Mathematical Formulation
Span-based QA:
$$P( ext{start} | ext{passage}, ext{question})$$
$$P( ext{end} | ext{passage}, ext{question})$$
Attention over passage:
$$ ext{score}_i = ext{question\_repr}^T ext{passage}_i$$
Answer extraction:
$$ ext{answer} = ext{argmax}_{s,e} P(s) \cdot P(e)$$
---
## Advanced Theory & Extensions
### Multi-hop Reasoning
Multiple passages; intermediate steps.
### Open-domain QA
Search + reading comprehension.
### Visual Question Answering
Image + text reasoning.
---
## Computational Considerations
Passage encoding: O(passage_len·hidden_dim).
Span scoring: O(passage_len²) for all spans.
Retrieval: O(corpus_size) for ranking.
---
## Practical Implementation Strategies
### Passage Encoding
BiDAF, DCN, or BERT-based.
### Answer Span Filtering
Confidence threshold; validity checks.
### Ensemble Methods
Multiple models; voting.
---
## Benchmark Datasets & Evaluation
SQuAD: Extractive QA.
MRQA: Multi-domain QA.
CoQA: Conversational QA.
---
## Key Challenges & Limitations
### Unanswerable Questions
Questions with no valid answer.
### Adversarial Examples
Robust to question perturbations.
### Long-range Dependencies
Reasoning over distant text.
---
## Hyperparameter Tuning
Passage length: 384-512 tokens.
Learning rate: 2e-5 to 5e-5.
Batch size: 16-32; memory.
---
## Real-World Applications & Case Studies
Search Engines: Answer extraction.
Reading Comprehension: Educational QA.
Information Assistants: Chatbot QA.
---
## Integration with Other Methods
QA + Retrieval → open-domain.
QA + Dialogue → conversational.
---
## Summary & Key Takeaways
Question Answering via span prediction enables machine reading comprehension through passage and question encoding with attention mechanism.
Principles:
1. Passage representation: context encoding.
2. Question representation: query encoding.
3. Span prediction: start and end positions.
4. Attention: focus on relevant passages.
5. Evaluation: EM and F1 metrics.
---
---
## Appendix: Practical Labs
### Lab 1: Span Extraction
import numpy as np
def extract_span(passage, start_idx, end_idx):
"""Extract answer span from passage"""
tokens = passage.split()
# Clamp indices
start_idx = max(0, min(start_idx, len(tokens) - 1))
end_idx = max(start_idx, min(end_idx, len(tokens) - 1))
# Extract span
answer_tokens = tokens[start_idx:end_idx + 1]
answer = " ".join(answer_tokens)
return answer
# Test
passage = "The quick brown fox jumps over the lazy dog"
answer = extract_span(passage, 1, 3)
assert answer == "quick brown fox", "Correct span extraction"
print("✓ Span extraction working")
if __name__ == "__main__":
print("Lab 1: SpanExtraction - PASSED")### Lab 2: Answer Score Computation
import numpy as np
def compute_answer_score(start_logits, end_logits, passage_len=100):
"""Compute scores for all possible answer spans"""
# Softmax for start and end
exp_start = np.exp(start_logits - np.max(start_logits))
start_probs = exp_start / exp_start.sum()
exp_end = np.exp(end_logits - np.max(end_logits))
end_probs = exp_end / exp_end.sum()
# Compute span scores
span_scores = np.zeros((passage_len, passage_len))
for start in range(passage_len):
for end in range(start, passage_len):
span_scores[start, end] = start_probs[start] * end_probs[end]
return span_scores
# Test
np.random.seed(42)
start_logits = np.random.randn(100)
end_logits = np.random.randn(100)
scores = compute_answer_score(start_logits, end_logits)
assert scores.shape == (100, 100), "Scores shape"
assert np.all(np.diag(scores, k=1) <= 1), "Span probabilities <= 1"
print("✓ Answer score working")
if __name__ == "__main__":
print("Lab 2: AnswerScore - PASSED")### Lab 3: Best Span Selection
import numpy as np
def select_best_span(span_scores, max_span_length=20):
"""Select best answer span"""
passage_len = span_scores.shape[0]
best_score = 0
best_start, best_end = 0, 0
for start in range(passage_len):
for end in range(start, min(start + max_span_length, passage_len)):
score = span_scores[start, end]
if score > best_score:
best_score = score
best_start = start
best_end = end
return best_start, best_end, best_score
# Test
np.random.seed(42)
span_scores = np.random.rand(100, 100)
# Make upper triangular
span_scores = np.triu(span_scores)
start, end, score = select_best_span(span_scores)
assert 0 <= start <= end < 100, "Valid span"
print("✓ Best span selection working")
if __name__ == "__main__":
print("Lab 3: BestSpanSelection - PASSED")### Lab 4: QA Metrics (EM and F1)
import numpy as np
def qa_metrics(prediction, reference):
"""Compute Exact Match and F1 for QA"""
pred_tokens = set(prediction.lower().split())
ref_tokens = set(reference.lower().split())
# Exact match
em = 1 if prediction.lower() == reference.lower() else 0
# F1
common = pred_tokens & ref_tokens
if len(common) == 0:
f1 = 0
else:
precision = len(common) / len(pred_tokens) if pred_tokens else 0
recall = len(common) / len(ref_tokens) if ref_tokens else 0
f1 = 2 * precision * recall / (precision + recall + 1e-8)
return em, f1
# Test
prediction = "the quick brown fox"
reference = "the quick brown fox"
em, f1 = qa_metrics(prediction, reference)
assert em == 1, "Exact match"
assert f1 == 1.0, "Perfect F1"
print("✓ QA metrics working")
if __name__ == "__main__":
print("Lab 4: QAMetrics - PASSED")