context carryover

**Context carryover** is the ability of a dialogue system to maintain and utilize information from **previous conversation turns** when processing new user messages. It is fundamental to creating natural, coherent multi-turn conversations rather than treating each message as an isolated query. **What Gets Carried Over** - **Entity References**: If a user says "Tell me about TSMC" then asks "What is their revenue?", the system must carry over that "their" refers to **TSMC**. - **Slot Values**: In task-oriented dialogue, previously stated preferences (cuisine, date, budget) persist across turns without the user needing to repeat them. - **Conversation Topic**: The current discussion topic provides implicit context for interpreting ambiguous queries. - **User Preferences**: Learned preferences and constraints from earlier in the conversation inform later responses. **Implementation Approaches** - **Full History**: Pass the entire conversation history to the LLM as context. Simple but limited by **context window size** and can become expensive for long conversations. - **Sliding Window**: Keep only the last N turns, discarding older history. Efficient but loses long-range context. - **Summarization**: Periodically summarize older conversation history into a compact representation, preserving key information while reducing token usage. - **Dialogue State Tracking**: Maintain a structured state object that captures all relevant information, independent of the raw conversation text. - **Memory Systems**: Use **vector databases** or other external memory to store and retrieve relevant past context on demand. **Challenges** - **Information Loss**: Summarization and windowing can lose critical details from earlier in the conversation. - **Topic Shifts**: Users may abruptly change topics, making older context irrelevant or even misleading. - **Ambiguity Resolution**: Determining what past context is relevant to the current turn requires sophisticated understanding. Effective context carryover is what separates a **truly conversational AI** from a simple question-answering system.