chain of thought

```svg ```ain-of-thought (CoT)** is the discovery that large language models solve hard problems far more reliably when they are prompted to reason step by step instead of blurting out an answer. Asking a model to "think it through" before answering — or simply appending "Let's think step by step" — can turn a wrong response into a right one on math, logic, and multi-step questions, with no change to the model's weights. This simple idea grew into an entire class of reasoning models that are explicitly trained to produce long internal reasoning before their final answer, and it reframed a key lever of AI capability: how much computation a model spends at inference time.\n\n```svg\n\n```\n\n**Why it works: hard problems need intermediate steps.** A single forward pass has a fixed amount of computation, and some problems genuinely require a sequence of dependent deductions that cannot be reached in one leap. Writing the reasoning out gives the model "scratch space" — each generated step becomes context the next step can build on, so the model effectively computes longer on harder inputs. The visible reasoning is not decoration; it is the mechanism by which extra computation happens.\n\n**Prompted CoT was the first form.** Few-shot chain-of-thought puts worked examples in the prompt showing the reasoning, and zero-shot CoT simply instructs the model to reason step by step. Both dramatically improved performance on benchmarks like grade-school and competition math, revealing that the capability was latent in the model all along and just needed to be elicited.\n\n**Reasoning models bake it in with reinforcement learning.** Rather than relying on the user to ask for reasoning, models like the o-series and R1-style systems are trained — often with RL that rewards correct final answers — to generate a long hidden "thinking" trace before responding. They learn to plan, check their work, backtrack, and try alternative approaches, and they allocate more thinking to harder problems. The reasoning trace may be hidden from the user, but it is where the real work happens.\n\n**This introduced test-time (inference-time) scaling.** For most of deep learning, capability came from scaling training — more data, more parameters, more pretraining compute. Reasoning models opened a second axis: spend more compute at inference by thinking longer, and accuracy keeps climbing on reasoning-heavy tasks. A single model can now be "turned up" for a hard problem by letting it think for longer, a fundamentally different cost and capability trade-off than picking a bigger model.\n\n**The trade-offs are latency, cost, and honesty.** Thinking tokens are generated tokens — they cost time and money, so reasoning models are slower and pricier per query, and are usually reserved for problems that need them. There is also an active question of faithfulness: the written reasoning does not always reflect the true computation that produced the answer, so a plausible-looking chain of thought is not a guarantee of a sound one.\n\n| Approach | How reasoning is triggered | Cost profile | Best for |\n|---|---|---|---|\n| Direct answer | none | cheapest, one pass | easy, factual, or lookup questions |\n| Prompted CoT | prompt asks for steps | a few extra tokens | many tasks, no special model |\n| Reasoning model | RL-trained to think first | variable, can be large | math, code, planning, hard logic |\n\nRead chain-of-thought through a *compute-you-spend-at-inference* lens rather than a *prompt-trick* lens: the deep shift is not the phrase "think step by step" but the realization that a model's answer quality on hard problems is a dial you can turn by letting it compute longer. Reasoning models productize that dial — trading tokens, latency, and dollars for accuracy — and add a second scaling axis to AI alongside the older one of simply training bigger.\n