verified reasoning

**Verified reasoning** is a framework for language model inference where **each reasoning step is independently checked for correctness** — using formal verification, execution, or trained verifier models to ensure that the chain of reasoning leading to an answer is logically valid, not just plausible-sounding. **Why Verified Reasoning?** - Standard LLM reasoning (including CoT) can produce **confident but wrong** intermediate steps — the model may skip logical steps, make arithmetic errors, or use invalid inference rules. - Without verification, there's no guarantee that a correct final answer was reached through correct reasoning — it may have arrived at the right answer by canceling errors or by lucky guessing. - **Verified reasoning** adds a checking mechanism to catch errors in intermediate steps before they propagate to the final answer. **Verification Methods** - **Execution-Based Verification**: Generate code for each reasoning step and execute it — the result must match the claimed intermediate result. - "47 × 83 = 3901" → Execute `47 * 83` → `3901` ✓ - Most reliable for mathematical and computational reasoning. - **Formal Verification**: Translate reasoning steps into formal logic and check with a theorem prover. - Each step must follow from previous steps by valid inference rules. - Used in mathematical proof verification (Lean, Coq, Isabelle). - **Trained Verifier Models (Process Reward Models)**: A separate neural network evaluates each reasoning step for correctness. - Trained on human annotations of step-by-step correctness. - Assigns a score to each step — steps below a threshold are flagged. - OpenAI's "Let's Verify Step by Step" demonstrated this approach. - **Self-Verification**: The same LLM is asked to check its own reasoning in a separate pass. - "Review the following reasoning. Is each step correct?" - Less reliable than external verification but requires no additional tools. - **Consensus Verification**: Generate multiple reasoning chains and check agreement. - If most chains agree on intermediate steps, those steps are likely correct. - Disagreement flags potentially erroneous steps for review. **Verified Reasoning Pipeline** 1. **Generate**: Model produces a step-by-step reasoning chain. 2. **Decompose**: Break the chain into individual steps. 3. **Verify Each Step**: Apply verification method to each step independently. 4. **Accept/Reject**: If all steps pass → accept the answer. If any step fails → regenerate from the failed step or choose an alternative chain. 5. **Output**: Return the verified answer along with the verified reasoning trace. **Applications** - **Mathematical Problem Solving**: Verify each calculation step — eliminate arithmetic and algebraic errors. - **Code Generation**: Execute generated code with test cases — verify functional correctness. - **Formal Proofs**: Each proof step verified by a proof assistant — guaranteeing mathematical validity. - **Scientific Reasoning**: Check that each logical inference follows from established premises and evidence. - **Safety-Critical Systems**: Any application where incorrect reasoning could have serious consequences. **Process vs. Outcome Supervision** - **Outcome Supervision**: Only check the final answer — rewards correct answers regardless of reasoning quality. - **Process Supervision**: Check each intermediate step — rewards correct reasoning even if the final answer has a transcription error. - Research shows **process supervision produces more reliable and aligned models** — the reasoning itself is incentivized to be correct. Verified reasoning is the **key to trustworthy AI reasoning** — by checking each step independently, it transforms speculative text generation into rigorous, verified inference.