retrieval augmented generation

**Retrieval-Augmented Generation (RAG)** is a **framework that enhances LLM outputs by retrieving relevant documents from a knowledge base and including them in the prompt** — combining parametric knowledge (model weights) with non-parametric knowledge (external documents). **RAG Architecture** 1. **Indexing**: Chunk documents → embed each chunk → store in vector database. 2. **Retrieval**: Embed the user query → find top-k most similar chunks by vector similarity. 3. **Augmentation**: Inject retrieved chunks into the LLM prompt as context. 4. **Generation**: LLM generates an answer grounded in the retrieved context. **Why RAG?** - **Reduces hallucination**: LLM answers from retrieved facts rather than generating from memory. - **Up-to-date knowledge**: Knowledge base can be updated without retraining the model. - **Attribution**: Can cite sources — users can verify which documents were used. - **Cost**: Cheaper than fine-tuning for knowledge-intensive tasks. **Key Components** - **Chunking Strategy**: Fixed size (512 tokens), sentence-based, or semantic chunking. - **Embedding Model**: OpenAI text-embedding-3, E5, GTE, BGE for dense retrieval. - **Vector Database**: Pinecone, Weaviate, Chroma, Qdrant, pgvector, FAISS. - **Reranking**: Cross-encoder reranker (Cohere Rerank, BGE-reranker) improves retrieval quality. **Advanced RAG Techniques** - **Hybrid Search**: Combine dense (semantic) + sparse (BM25 keyword) retrieval. - **HyDE (Hypothetical Document Embeddings)**: Generate a hypothetical answer first, then retrieve. - **Self-RAG**: Model decides when to retrieve and evaluates retrieved passages. - **Multi-hop RAG**: Iterative retrieval for complex multi-step questions. **RAG vs. Fine-tuning**: RAG is preferred for dynamic or large knowledge bases; fine-tuning is better for style, format, and capability changes. RAG is **the standard architecture for enterprise LLM applications** — it bridges the gap between general-purpose LLMs and domain-specific knowledge requirements.