Retrieval-Augmented Generation (RAG) is the dominant pattern for making a language model answer questions over knowledge it was never trained on — your company's documents, a product manual, last week's tickets. Instead of hoping the answer is baked into the model's frozen weights, RAG retrieves the relevant passages at query time and hands them to the model as context, so the model reasons over fresh, specific, citable text rather than its hazy parametric memory. It is how most enterprise "chat with your docs" systems work, and the standard alternative to fine-tuning when the goal is to inject knowledge rather than change behavior.\n\n``svg\n\n``\n\nThe knowledge is prepared offline, once. Source documents are split into passages ("chunks"), each chunk is converted to a vector by an embedding model, and the vectors are stored in a vector database. Because embeddings place semantically similar text near each other, this index becomes a searchable map of meaning — you can later find passages by what they mean, not just by keyword overlap. Chunking strategy (size, overlap, boundaries) quietly determines much of a RAG system's quality.\n\nEach query runs a fast retrieve-then-generate loop. The user's question is embedded with the same model, the vector database returns the top-k nearest passages, those passages are pasted into the prompt alongside the question, and the LLM generates an answer grounded in them. The model's weights are never touched; all the domain knowledge arrives through the context window at inference time.\n\nRAG's headline benefit is fresh, updatable, attributable knowledge. Add or edit a document and the very next answer reflects it — no retraining, no fine-tuning run. Because the model is answering from retrieved passages, it can cite them, which makes answers auditable and dramatically reduces hallucination on factual questions. This is why RAG, not fine-tuning, is the usual first choice for question-answering over a changing corpus.\n\nRetrieval quality is the whole ballgame. If the right passage is not retrieved, the model cannot use it, and it may confidently fill the gap with a fabrication. Production systems therefore invest heavily in retrieval: hybrid search that blends vector similarity with keyword (BM25) matching, a reranker that reorders candidates with a heavier model, query rewriting, and metadata filtering. "Garbage retrieved, garbage generated" is the operative failure mode.\n\nRAG and fine-tuning solve different problems. Fine-tuning changes how the model behaves — tone, format, skills — by adjusting weights. RAG changes what the model knows at answer time by adjusting context. They compose well: fine-tune a model to follow your answer format and use retrieval to feed it current facts. Reach for RAG when knowledge is large, private, or changing; reach for fine-tuning when you need new behavior or style.\n\n| Dimension | RAG | Fine-tuning |\n|---|---|---|\n| Changes | the context (retrieved text) | the weights |\n| Best for | injecting fresh/private knowledge | changing behavior, tone, format |\n| Update cost | edit a document, re-embed it | run another training job |\n| Attribution | can cite retrieved sources | opaque, no citations |\n| Main failure | wrong passages retrieved | catastrophic forgetting, staleness |\n\nRead RAG through a retrieval-quality lens rather than a bigger-model lens: the generator is rarely the bottleneck — a strong model given the wrong passages still answers wrong, while a modest model given exactly the right passage answers correctly and cites it. Almost all of the engineering payoff in a RAG system lives upstream of the LLM, in how documents are chunked, embedded, searched, reranked, and filtered before a single token is generated.\n