Question Decomposition is the advanced reasoning technique that breaks complex questions into simpler sub-questions to enable systematic multi-step problem solving — allowing language models to tackle problems that require combining multiple pieces of information, performing sequential reasoning steps, or synthesizing knowledge from different domains by addressing each component independently before combining results.

What Is Question Decomposition?

• Definition: A prompting and reasoning strategy that transforms a single complex question into a chain of simpler, answerable sub-questions.
• Core Principle: Complex problems become tractable when decomposed into manageable steps that can be solved independently.
• Key Mechanism: Each sub-question's answer feeds into subsequent questions, building toward the final comprehensive answer.
• Relationship to CoT: Extends chain-of-thought prompting by explicitly structuring the reasoning into discrete retrievable questions.

Why Question Decomposition Matters

• Improved Accuracy: Models answer simple questions more reliably than complex multi-hop ones, so decomposition improves overall correctness.
• Transparent Reasoning: Each sub-question and answer is visible, making the reasoning chain auditable and debuggable.
• Better Retrieval: Simple sub-questions match document content more precisely than complex compound queries in RAG systems.
• Error Isolation: When answers are wrong, decomposition reveals exactly which reasoning step failed.
• Scalability: Arbitrarily complex questions can be handled by decomposing into sufficiently simple components.

How Question Decomposition Works

Step 1 — Identify Information Needs:

• Parse the complex question to identify distinct pieces of information required.
• Map dependencies between sub-questions (which answers are needed before others can be asked).

Step 2 — Generate Sub-Questions:

• Create focused, answerable sub-questions for each information need.
• Order sub-questions to respect dependencies and build reasoning incrementally.

Step 3 — Solve and Synthesize:

• Answer each sub-question independently using retrieval or reasoning.
• Combine sub-answers into a coherent final response addressing the original complex question.

Decomposition Strategies

Tools & Applications

• RAG Systems: Decomposed queries retrieve more relevant documents than monolithic complex queries.
• Multi-Hop QA: Benchmarks like HotpotQA and MuSiQue specifically test decomposition capabilities.
• Research Agents: AI agents use decomposition to plan multi-step research workflows.
• Education: Teaching systems decompose student questions to provide step-by-step explanations.

Question Decomposition is fundamental to building AI systems capable of complex reasoning — transforming intractable multi-hop problems into manageable chains of simple questions that models can answer reliably and transparently.