Reranking for Better Retrieval

What is Reranking? Reranking is a two-stage retrieval process: first retrieve many candidates quickly (using vector search), then rerank them for relevance using a more accurate model.

Why Rerank?

Two-Stage Pipeline

Query
  |
  v
[Bi-encoder retrieval] (top 100)
  |
  v
[Cross-encoder reranking]
  |
  v
[Top 10 most relevant results]

Cross-Encoder vs Bi-Encoder

Bi-Encoder (Fast) Encode query and documents separately:

query_embedding = embed(query)
doc_embeddings = [embed(doc) for doc in docs]
scores = cosine_similarity(query_embedding, doc_embeddings)

Cross-Encoder (Accurate) Encode query and document together:

# Sees full context, can understand relationships
score = cross_encoder.predict([query, document])

Popular Rerankers

Implementation

from sentence_transformers import CrossEncoder

# Load reranker
reranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")

# First stage: vector retrieval
candidates = vector_store.query(query, top_k=100)

# Second stage: reranking
pairs = [[query, doc.text] for doc in candidates]
scores = reranker.predict(pairs)

# Sort by reranker scores
reranked = sorted(zip(candidates, scores), key=lambda x: x[1], reverse=True)
top_results = reranked[:10]

When to Use Reranking

Performance Tips

• Retrieve more candidates than final need (100 or 50 for top 10)
• Consider reranker latency in architecture
• Batch reranking calls where possible
• Cache reranking for repeated queries