Dense-sparse hybrid retrieval combines two fundamentally different search approaches — dense (neural) retrieval using vector embeddings and sparse (keyword) retrieval using traditional term-matching algorithms — to achieve more robust and comprehensive search results in RAG and information retrieval systems.

The Two Components

• Dense Retrieval: Uses a neural encoder (like BERT, E5, or BGE) to convert queries and documents into dense vector embeddings. Retrieval is based on semantic similarity (cosine similarity or dot product) in the embedding space. Great for understanding meaning and paraphrases.
• Sparse Retrieval: Uses algorithms like BM25 or TF-IDF that represent documents as sparse vectors based on term frequency. Retrieval is based on exact keyword matching. Great for specific terms, names, codes, and rare words.

Why Hybrid Works Better

• Dense Strengths: Understands that "automobile" and "car" are related, captures contextual meaning, handles paraphrases and conceptual queries.
• Dense Weaknesses: Can miss exact keyword matches, struggles with rare terms, codes, and proper nouns.
• Sparse Strengths: Perfect for exact term matching, handles rare/technical vocabulary, fast and interpretable.
• Sparse Weaknesses: Misses synonyms and semantic relationships, no understanding of meaning.

Fusion Methods

• RRF (Reciprocal Rank Fusion): Merge rankings by position — simple and effective.
• Weighted Score Fusion: Combine normalized scores with tunable weights (e.g., 0.7 × dense + 0.3 × sparse).
• Learned Fusion: Train a model to optimally combine scores based on query type.

Production Implementations

Major vector databases support hybrid search: Pinecone (sparse-dense vectors), Weaviate (hybrid search), Elasticsearch (kNN + BM25), and Qdrant (sparse vectors). Hybrid retrieval consistently outperforms either approach alone across diverse benchmarks and is considered a best practice for production RAG systems.