re-ranking

Re-ranking is a second-stage retrieval step in RAG pipelines that rescores initially retrieved documents using a more powerful cross-encoder model, significantly improving relevance ranking compared to the first-stage bi-encoder retrieval. Two-stage pipeline: (1) initial retrieval (bi-encoder—encode query and documents independently, fast ANN search over millions of documents, returns top-k candidates, typically k=50-100), (2) re-ranking (cross-encoder—jointly encodes query and each candidate document through full transformer attention, produces relevance score, reorders top-k). Why cross-encoders are better: bi-encoders compute query and document embeddings independently (no cross-attention), missing fine-grained query-document interactions. Cross-encoders process [query, document] pairs jointly, capturing token-level relevance signals. Models: (1) Cohere Rerank (API-based, multilingual), (2) BGE Reranker (open-source), (3) cross-encoder/ms-marco (sentence-transformers), (4) ColBERT (late interaction—faster than full cross-encoder with similar quality), (5) RankGPT/LLM-based rerankers (use LLM to judge relevance). Performance: re-ranking typically improves NDCG@10 by 5-15% over bi-encoder retrieval alone. Latency: cross-encoder processes k candidates sequentially (not indexed)—latency = k × inference_time. Typical: 50 candidates × 5ms = 250ms. Optimization: (1) reduce k (fewer candidates to re-rank), (2) use distilled rerankers (smaller, faster models), (3) cache frequent queries. Integration: LangChain, LlamaIndex, and Haystack all support re-ranking stages. Essential component for production RAG systems where retrieval quality directly impacts generation accuracy.