context window
A context window is the maximum amount of text, measured in tokens, that a language model can consider at once when generating a response, encompassing everything from the conversation history to any documents or instructions provided alongside the current question.
```flowchart
{
"rows": [
{ "type": "nodes", "items": [
{ "title": "Conversation, documents, and instructions provided", "sub": "all compete for space within one fixed limit", "tone": "neutral" }
]},
{ "type": "arrow" },
{ "type": "group", "title": "Context window holds everything the model can see", "items": [
{ "title": "Measured in tokens, with a fixed maximum size", "sub": "content beyond the limit isn't visible to the model", "tone": "blue" }
]},
{ "type": "arrow" },
{ "type": "nodes", "items": [
{ "title": "Model generates its response from what fits", "sub": "larger windows allow longer documents and conversations", "tone": "green" }
]}
]
}
```
**A context window exists because a language model doesn't have unlimited working memory — it can only directly consider a fixed maximum amount of text at any one time when generating its response.** Everything relevant to a given request, including the ongoing conversation history, any uploaded documents, and specific instructions, all has to fit within this fixed context window, measured in tokens rather than raw characters or words; content that doesn't fit within that limit simply isn't visible to the model when it generates its response.
```svg
```
```svg
```
| Aspect | Small context window | Large context window |
|---|---|---|
| Conversation length supported | Shorter before older parts drop off | Much longer conversations retained |
| Document length supported | Short documents only | Long documents can fit entirely |
| Computational cost | Lower | Generally higher |
| Common use | Simple, short interactions | Long documents, extended conversations |
**Context window size is typically measured in tokens rather than words or characters, since a token, a model's basic unit of text, doesn't correspond to a fixed word or character count.** Because a single word can sometimes be split into multiple tokens or several short words can combine into fewer tokens depending on the model's specific tokenization scheme, context window limits are expressed in tokens rather than a more intuitive but less precise word or character count, making the actual amount of usable text somewhat variable depending on the specific content involved.
**When a conversation or document exceeds the available context window, older or less relevant content typically has to be dropped, summarized, or otherwise managed to make room.** Since a context window has a hard maximum size, a conversation or document collection that exceeds it can't simply keep growing indefinitely — systems built around language models often need specific strategies, such as summarizing older conversation turns or selectively including only the most relevant portions of a large document, to manage content that exceeds the available context window.
**Larger context windows generally enable working with longer documents and more extended conversations, but often come with increased computational cost for processing that additional context.** As context window sizes have grown across successive model generations, enabling genuinely long documents and extended conversations to be handled directly, this increased capability has generally come with a real computational cost, since processing a longer context window typically requires proportionally more computation than a shorter one.
Read the context window through a desk-space lens: no matter how capable the person working at that desk is, they can only directly work with what actually fits on it at one time — the context window is exactly that desk's fixed size, and anything that doesn't fit simply isn't part of what's immediately being considered.