In-context learning is the ability of a large language model to pick up a brand-new task purely from examples or instructions placed in its prompt, at inference time, without a single change to its weights. Show a model three examples of English-to-French translation and then a fourth English word, and it will translate it — not because anyone retrained it, but because the pattern in the context steered its next-token prediction. This is the capability that turned language models from things you fine-tune into things you simply prompt, and understanding it means separating two ideas that sound identical but are not: "learning" in the sense of updating parameters, and "learning" in the sense of conditioning on context. In-context learning is entirely the second kind.\n\nThe weights never change — the "learning" happens inside a single forward pass. Ordinary machine learning improves a model by computing gradients and nudging parameters; in-context learning does none of that. The model is frozen. All that happens is that the examples in the prompt become part of the input the attention mechanism reads, and that context reshapes the probability distribution over the next token. The task-specific "adaptation" lives in the activations flowing through the network for that one query and vanishes the moment the prompt ends. This is why in-context learning is instant and cheap — no training run, no data pipeline — and also why it is fragile: nothing is saved, so the model relearns the task from scratch on every call.\n\nIt is an emergent behavior of scale, mechanistically linked to induction heads. Small models barely do in-context learning; the ability strengthens sharply as models grow, which is why it is often cited as a hallmark emergent capability. Mechanistic interpretability has traced a large part of it to induction heads — attention circuits that form during training and implement a simple but powerful rule: find where the current token appeared earlier in the context, look at what came next, and predict that same continuation. This copy-and-continue machinery is what lets a model notice the "input → output" rhythm of your examples and extend it, and its emergence during training coincides with a visible jump in in-context learning ability.\n\nZero-, one-, and few-shot are points on one axis, and the whole thing is powerful but brittle. The number of demonstrations you provide — zero, one, or a handful — are just settings of the same in-context mechanism, trading prompt length for reliability. But because the model is reading surface patterns rather than truly understanding the task, in-context learning is notoriously sensitive: the order of your examples, their format, even which labels you use can swing accuracy substantially, and studies have shown models sometimes lean on the format and label distribution more than the actual input-output mapping. This is the double-edged nature of the capability — it makes models astonishingly flexible with no training, yet the same reliance on context patterns makes results unstable in ways that fine-tuning, which bakes the task into the weights, is not.\n\n| Property | In-context learning |\n|---|---|\n| What changes | Nothing — weights are frozen |\n| Where "learning" lives | Activations of a single forward pass |\n| Mechanism | Attention over prompt; induction heads |\n| Appears | Emergently, and grows with model scale |\n| Strength | Instant task adaptation, no retraining |\n| Weakness | Brittle to example order, format, labels |\n\n``svg\n\n``\n\nThe unhelpful way to think about in-context learning is as a mysterious flash of intelligence, as if the model briefly "trained itself" on your examples. The useful way is to hold onto the one hard fact that nothing about the model changed: the weights are frozen, and every bit of task-specific adaptation lives in the activations of a single forward pass, produced by attention reading your prompt and induction heads extending its patterns. That single fact explains everything downstream — why it is instant and needs no training run, why it emerged only at scale, why zero-, one-, and few-shot are just dials on the same mechanism, and why it is so sensitive to the order and format of your examples. Read in-context learning through a learning-that-happens-in-the-forward-pass lens rather than an another-training-step lens, and its power and its fragility stop being a paradox and become two faces of the same simple truth.