reranking
**Reranking for Better Retrieval**
**What is Reranking?**
Reranking is a two-stage retrieval process: first retrieve many candidates quickly (using vector search), then rerank them for relevance using a more accurate model.
**Why Rerank?**
| Approach | Speed | Accuracy | Use |
|----------|-------|----------|-----|
| Bi-encoder (embedding) | Fast | Good | First retrieval |
| Cross-encoder (reranker) | Slow | Better | Rerank top-k |
**Two-Stage Pipeline**
```
Query
|
v
[Bi-encoder retrieval] (top 100)
|
v
[Cross-encoder reranking]
|
v
[Top 10 most relevant results]
```
**Cross-Encoder vs Bi-Encoder**
**Bi-Encoder (Fast)**
Encode query and documents separately:
```python
query_embedding = embed(query)
doc_embeddings = [embed(doc) for doc in docs]
scores = cosine_similarity(query_embedding, doc_embeddings)
```
**Cross-Encoder (Accurate)**
Encode query and document together:
```python
# Sees full context, can understand relationships
score = cross_encoder.predict([query, document])
```
**Popular Rerankers**
| Model | Type | Highlights |
|-------|------|------------|
| Cohere Rerank | API | Commercial, excellent quality |
| bge-reranker | Open | Various sizes, multilingual |
| cross-encoder/ms-marco | Open | Strong baseline |
| mixedbread-ai/mxbai-rerank | Open | State-of-the-art open |
**Implementation**
```python
from sentence_transformers import CrossEncoder
# Load reranker
reranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")
# First stage: vector retrieval
candidates = vector_store.query(query, top_k=100)
# Second stage: reranking
pairs = [[query, doc.text] for doc in candidates]
scores = reranker.predict(pairs)
# Sort by reranker scores
reranked = sorted(zip(candidates, scores), key=lambda x: x[1], reverse=True)
top_results = reranked[:10]
```
**When to Use Reranking**
| Scenario | Recommendation |
|----------|----------------|
| High precision needed | Always rerank |
| Latency critical | Skip or use fast reranker |
| Large candidate pool | Essential |
| Domain-specific | Fine-tune reranker |
**Performance Tips**
- Retrieve more candidates than final need (100 or 50 for top 10)
- Consider reranker latency in architecture
- Batch reranking calls where possible
- Cache reranking for repeated queries