conversation
**Multi-Turn Conversations** are the **stateless simulation of persistent dialogue achieved by including complete conversation history in every API call** — requiring developers to explicitly manage conversation state, context window budgets, and history truncation strategies because language models have no built-in memory between API calls and must reconstruct context from the provided message array on every request.
**What Is a Multi-Turn Conversation?**
- **Definition**: A sequence of alternating user and assistant messages where each turn builds on prior context — the AI remembers what was said, refers to previous topics, and maintains coherent dialogue across multiple exchanges.
- **The Fundamental Illusion**: LLMs are stateless functions — f(messages) → response. They have no memory, no session state, no persistent knowledge of previous calls. Every "memory" in a conversation is achieved by re-sending the entire history.
- **Developer Responsibility**: Unlike traditional databases that persist state automatically, multi-turn AI conversations require the application layer to explicitly manage, store, and re-transmit conversation history on every turn.
- **Context Window Budget**: The conversation history consumes the model's context window — a 128K token model can hold roughly 90,000-100,000 tokens of conversation before history must be pruned.
**Why Multi-Turn Conversation Management Matters**
- **Coherence**: Without proper history management, models cannot refer to earlier parts of the conversation, answer follow-up questions correctly, or maintain consistent persona and decisions.
- **Cost**: Each turn re-sends the entire history — a 10-turn conversation at turn 10 sends 9x the tokens of turn 1. Input token costs compound multiplicatively.
- **Latency**: Longer context windows take longer to process — first-token latency increases with conversation length.
- **Context Window Limits**: 4K, 8K, 32K, 128K token limits constrain how much history can be maintained — requiring management strategies for long conversations.
- **Relevance Decay**: Early conversation turns may become irrelevant as conversation evolves — naive FIFO truncation drops important early context (user's initial problem statement).
**Multi-Turn Implementation Pattern**
```python
conversation_history = []
def chat(user_message: str, system_prompt: str) -> str:
# Add user message to history
conversation_history.append({"role": "user", "content": user_message})
# Build complete message array (system + full history)
messages = [{"role": "system", "content": system_prompt}] + conversation_history
# Call API with full history
response = client.chat.completions.create(
model="gpt-4o",
messages=messages
)
# Extract and store assistant response
assistant_message = response.choices[0].message.content
conversation_history.append({"role": "assistant", "content": assistant_message})
return assistant_message
```
**Context Management Strategies**
**Naive Truncation (FIFO)**:
- Drop oldest messages when context window fills.
- Simple to implement, but loses critical early context (initial problem statement, user preferences).
- Best for: simple Q&A sessions without complex dependencies.
**Smart Truncation (Preserve Anchors)**:
- Always keep: system prompt + first user message + last N turns.
- Drop: middle turns when context fills.
- Better for: conversations with important setup context in early turns.
**Summarization**:
- When history exceeds threshold, summarize old turns: "Summarize this conversation in 200 words preserving key decisions and context."
- Insert summary as system context; discard summarized turns.
- Best for: long conversations where summarized context suffices.
**Vector Memory**:
- Store all turns as embeddings in a vector database.
- On each turn, retrieve the K most semantically relevant prior turns.
- Inject retrieved context into the current prompt.
- Best for: very long sessions (days/weeks) where exact history retrieval is too large for context.
**Context Window Usage by Model**
| Model | Context Window | ~Turns at 500 tok/turn |
|-------|---------------|----------------------|
| GPT-4o mini | 128K | ~256 turns |
| GPT-4o | 128K | ~256 turns |
| Claude 3.5 Sonnet | 200K | ~400 turns |
| Gemini 1.5 Pro | 1M | ~2,000 turns |
| Llama 3.1 8B | 128K | ~256 turns |
**Token Cost Implications**
In a 20-turn conversation with 200 tokens per turn:
- Turn 1: 200 input tokens
- Turn 10: 2,000 input tokens (full history)
- Turn 20: 4,000 input tokens (full history)
- Total input tokens: ~42,000 (sum of 200+400+...+4000)
At GPT-4o pricing ($5/1M input tokens): ~$0.21 for a 20-turn conversation — manageable, but in production systems with thousands of concurrent conversations, these costs compound.
Multi-turn conversations are **the foundational interaction paradigm for AI assistants** — but beneath the seamless dialogue experience lies a stateless function repeatedly consuming growing context windows, and managing this architecture efficiently — through smart truncation, summarization, and vector memory — is what separates prototype chatbots from production-grade AI applications.