contextual embeddings

Contextual embeddings incorporate surrounding context to generate more accurate document representations. **Problem**: Standard chunking embeds each chunk in isolation, losing document-level and positional context. A chunk about "the process" may be ambiguous without knowing what document it's from. **Solutions**: **Prepend context**: Add document title, section headers, or summary to each chunk before embedding. **Contextual embedding models**: Train embeddings that consider surrounding text. **Late contextualization**: Retrieve chunks, inject parent context at generation time. **Implementation**: For each chunk, prepend "Document: {title}. Section: {section}. Content: {chunk}" then embed. **Anthropic's approach**: Prepend LLM-generated chunk summary that situates the chunk in document context. **Benefits**: Resolves ambiguous references, improves retrieval relevance, particularly for structured documents. **Trade-offs**: Longer text to embed (cost, potential truncation), preprocessing overhead. **Use cases**: Technical documentation with sections, legal documents, any content with document-level context. **Results**: Significant retrieval improvements (20-30% on some benchmarks), especially for out-of-context chunks.