Text Chunking for RAG

Why Chunking Matters RAG systems need to split documents into smaller pieces for embedding and retrieval. Chunk size and strategy significantly impact retrieval quality.

Chunking Strategies

Fixed Size Split by character/token count:

def fixed_chunk(text: str, chunk_size: int = 500, overlap: int = 50) -> list:
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks

Semantic Chunking Split at natural boundaries:

• Paragraphs
• Sections (headers)
• Sentences
• Topics (using embeddings)

Recursive Splitting Try multiple separators hierarchically:

from langchain.text_splitter import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=500,
    chunk_overlap=50,
    separators=["

", "
", ". ", " ", ""]
)
chunks = splitter.split_text(document)

Chunk Size Guidelines

Overlap Considerations

Document-Specific Chunking

Code

def chunk_code(code: str) -> list:
    # Split by function/class definitions
    # Keep docstrings with their functions
    # Respect indentation boundaries

Markdown

def chunk_markdown(md: str) -> list:
    # Split at headers
    # Keep header hierarchy metadata
    # Preserve code blocks intact

Tables Keep tables together:

def handle_table(table_text: str) -> list:
    # Never split a table
    # Include table caption
    # Add column headers to each chunk if splitting rows

Metadata Attach context to chunks:

chunk = {
    "text": "...",
    "source": "document.pdf",
    "page": 5,
    "section": "Introduction",
    "char_start": 1500,
    "char_end": 2000
}

Metadata enables filtering, citation, and context reconstruction.