dense-sparse hybrid retrieval
**Dense-sparse hybrid retrieval** combines two fundamentally different search approaches — **dense (neural) retrieval** using vector embeddings and **sparse (keyword) retrieval** using traditional term-matching algorithms — to achieve more robust and comprehensive search results in **RAG** and information retrieval systems.
**The Two Components**
- **Dense Retrieval**: Uses a neural encoder (like **BERT, E5, or BGE**) to convert queries and documents into **dense vector embeddings**. Retrieval is based on **semantic similarity** (cosine similarity or dot product) in the embedding space. Great for understanding meaning and paraphrases.
- **Sparse Retrieval**: Uses algorithms like **BM25** or **TF-IDF** that represent documents as **sparse vectors** based on term frequency. Retrieval is based on **exact keyword matching**. Great for specific terms, names, codes, and rare words.
**Why Hybrid Works Better**
- **Dense Strengths**: Understands that "automobile" and "car" are related, captures contextual meaning, handles paraphrases and conceptual queries.
- **Dense Weaknesses**: Can miss exact keyword matches, struggles with rare terms, codes, and proper nouns.
- **Sparse Strengths**: Perfect for exact term matching, handles rare/technical vocabulary, fast and interpretable.
- **Sparse Weaknesses**: Misses synonyms and semantic relationships, no understanding of meaning.
**Fusion Methods**
- **RRF (Reciprocal Rank Fusion)**: Merge rankings by position — simple and effective.
- **Weighted Score Fusion**: Combine normalized scores with tunable weights (e.g., 0.7 × dense + 0.3 × sparse).
- **Learned Fusion**: Train a model to optimally combine scores based on query type.
**Production Implementations**
Major vector databases support hybrid search: **Pinecone** (sparse-dense vectors), **Weaviate** (hybrid search), **Elasticsearch** (kNN + BM25), and **Qdrant** (sparse vectors). Hybrid retrieval consistently outperforms either approach alone across diverse benchmarks and is considered a **best practice** for production RAG systems.