what is retrieval-augmented generation
Retrieval-augmented generation, or RAG, is a technique that combines a language model with an external search step, retrieving relevant documents or facts from an outside knowledge source and feeding them to the model as context before it generates its final answer.
```flowchart
{
"rows": [
{ "type": "nodes", "items": [
{ "title": "User asks a question", "sub": "language model alone may lack current or specific facts", "tone": "neutral" }
]},
{ "type": "arrow" },
{ "type": "group", "title": "Retrieval step searches an external source", "items": [
{ "title": "Relevant documents or facts pulled in as context", "sub": "grounds the answer in real, up-to-date information", "tone": "blue" }
]},
{ "type": "arrow" },
{ "type": "nodes", "items": [
{ "title": "Model generates an answer using that retrieved context", "sub": "more accurate and verifiable than relying on memory alone", "tone": "green" }
]}
]
}
```
**RAG exists because a language model's built-in knowledge is fixed at training time and can't include everything relevant, especially fast-changing or highly specific information a user might need.** Rather than relying solely on whatever a language model happened to learn during training, RAG first retrieves relevant documents or facts from an external knowledge source, such as a document database or search index, and provides that retrieved material to the model directly as context, letting it generate a response grounded in current, specific, and verifiable information rather than relying purely on its own fixed internal training.
```svg
```
```svg
```
| Aspect | Model alone | Retrieval-augmented generation |
|---|---|---|
| Knowledge source | Fixed at training time | External source, searchable and updatable |
| Currency of information | Can become outdated | As current as the retrieved source |
| Verifiability | Harder to trace claims | Retrieved sources can be cited |
| Common use | General knowledge questions | Domain-specific, current, or private data |
**RAG's retrieval step commonly relies on converting both the query and candidate documents into numerical representations, then finding the closest matches, rather than simple keyword search alone.** Many RAG systems use embedding-based retrieval, which converts the user's query and candidate documents into numerical vector representations capturing their meaning, then finds the documents whose vectors are most similar to the query's — this approach can surface genuinely relevant documents even when they don't share exact keywords with the original query.
**RAG lets an organization ground a language model's answers in their own private or specialized documents, without needing to retrain the underlying model itself.** Because RAG's retrieval source can be any searchable document collection, including an organization's own internal documents, RAG offers a practical way to have a language model answer questions grounded in specific, private, or specialized information without the significant cost and complexity of retraining the model on that data directly.
**RAG system quality depends heavily on retrieval quality, since even a highly capable language model will struggle to give a good answer if it's given irrelevant or poor-quality retrieved context.** If the retrieval step surfaces irrelevant, incomplete, or low-quality documents, the language model's final generated answer is likely to suffer regardless of how capable the underlying model otherwise is — this dependency is why retrieval quality receives significant engineering attention in real-world RAG system design.
Read retrieval-augmented generation through an open-book-exam lens: rather than answering purely from memory, the model first looks up relevant material specific to the question at hand, then writes its answer using that material as a reference — producing responses that are typically more accurate and more grounded in verifiable, current information.