Query Expansion and Rewriting

Why Expand Queries? User queries are often short, ambiguous, or miss relevant terminology. Query expansion improves retrieval by adding related terms or reformulating the query.

Expansion Techniques

Synonym Expansion

def expand_with_synonyms(query: str) -> str:
    expanded = llm.generate(f"""
Add synonyms and related terms to this search query.
Keep the original query and add alternatives.

Query: {query}
Expanded:
    """)
    return expanded

LLM Query Rewriting

def rewrite_query(query: str) -> str:
    rewritten = llm.generate(f"""
Rewrite this query to be more specific and detailed for search:
"{query}"

Consider:
- What the user is really asking
- Related technical terms
- Alternative phrasings

Rewritten query:
    """)
    return rewritten

Multi-Query Generation Generate multiple queries to cover different aspects:

def multi_query(query: str) -> list:
    queries = llm.generate(f"""
Generate 3 different search queries that would help answer:
"{query}"

1.
2.
3.
    """)
    return parse_queries(queries)

Query Decomposition Break complex queries into sub-queries:

Original: "Compare Python and Rust for web development performance"
Sub-queries:
1. "Python web framework performance benchmarks"
2. "Rust web framework performance benchmarks"
3. "Python vs Rust async performance"

Fusion Strategies Combine results from multiple queries:

Reciprocal Rank Fusion (RRF)

def rrf_combine(results_lists: list) -> list:
    scores = {}
    for results in results_lists:
        for rank, doc in enumerate(results):
            scores[doc] = scores.get(doc, 0) + 1/(60 + rank)
    return sorted(scores.items(), key=lambda x: x[1], reverse=True)

When to Use

Practical Tips

• Dont over-expand (noise drowns signal)
• Use domain-specific expansion prompts
• Consider query classification first
• Cache expansions for common queries