Context carryover is the ability of a dialogue system to maintain and utilize information from previous conversation turns when processing new user messages. It is fundamental to creating natural, coherent multi-turn conversations rather than treating each message as an isolated query.

What Gets Carried Over

• Entity References: If a user says "Tell me about TSMC" then asks "What is their revenue?", the system must carry over that "their" refers to TSMC.
• Slot Values: In task-oriented dialogue, previously stated preferences (cuisine, date, budget) persist across turns without the user needing to repeat them.
• Conversation Topic: The current discussion topic provides implicit context for interpreting ambiguous queries.
• User Preferences: Learned preferences and constraints from earlier in the conversation inform later responses.

Implementation Approaches

• Full History: Pass the entire conversation history to the LLM as context. Simple but limited by context window size and can become expensive for long conversations.
• Sliding Window: Keep only the last N turns, discarding older history. Efficient but loses long-range context.
• Summarization: Periodically summarize older conversation history into a compact representation, preserving key information while reducing token usage.
• Dialogue State Tracking: Maintain a structured state object that captures all relevant information, independent of the raw conversation text.
• Memory Systems: Use vector databases or other external memory to store and retrieve relevant past context on demand.

Challenges

• Information Loss: Summarization and windowing can lose critical details from earlier in the conversation.
• Topic Shifts: Users may abruptly change topics, making older context irrelevant or even misleading.
• Ambiguity Resolution: Determining what past context is relevant to the current turn requires sophisticated understanding.

Effective context carryover is what separates a truly conversational AI from a simple question-answering system.