Hebbian Learning
Hebbian learning is the principle that a connection between two neurons is strengthened when one repeatedly takes part in making the other fire, so that correlated activity leaves a lasting trace in the wiring. It is named for the psychologist Donald Hebb, who stated the idea in his 1949 book The Organization of Behavior as a hypothesis about how a cell's efficiency in exciting its neighbor could grow through persistent joint activity, and it remains the best-known example of a local learning rule: one that changes a connection using only information available at the two ends of that connection.
The famous slogan is a later paraphrase, not Hebb's own wording. Hebb's postulate said, in effect, that if cell A is near enough to cell B to help excite it, and does so repeatedly or persistently, some growth process or metabolic change occurs in one or both cells that raises A's effectiveness at firing B. The compact version, cells that fire together wire together, is usually credited to the neuroscientist Carla Shatz, who summarized it that way in 1992. The distinction matters because Hebb's text stresses causation — A has to take part in firing B — and not mere coincidence, an emphasis that the cruder slogan loses and that spike-timing experiments later brought back into focus.
In its simplest mathematical form the rule has one term. Take a connection weight w between an input of activity x and an output of activity y; the Hebbian update adds to w an amount proportional to the product of x and y, so the change is the learning rate times x times y. Inputs and outputs that are active together strengthen their link, and ones that are not leave it alone. The rule yields an associative memory almost immediately: summing such outer products over stored patterns builds a weight matrix that lets a partial or noisy pattern recall a whole one, which is how John Hopfield's 1982 network stores its memories, and in 1955–56 Nathaniel Rochester, John Holland, L. H. Haibt and W. L. Duda ran one of the earliest computer simulations of Hebbian cell assemblies on an IBM 701.
Pure Hebbian growth is unstable, and the fixes defined much of the later theory. Since strengthening raises activity, which raises strengthening, weights under the plain rule grow without limit. Erkki Oja's 1982 rule adds a term that subtracts a multiple of the output squared times the weight, which keeps the weight vector at unit length and, remarkably, makes a single linear neuron converge to the first principal component of its input data, linking a biological idea to a classic statistical method. Elie Bienenstock, Leon Cooper and Paul Munro's 1982 model instead uses a sliding threshold, so that moderate activity weakens connections and high activity strengthens them. Spike-timing-dependent plasticity (STDP), reported in experiments by Henry Markram and colleagues in 1997 and by Guo-Qiang Bi and Mu-Ming Poo in 1998, restores Hebb's causal asymmetry: when the input spike arrives shortly before the output spike the synapse is potentiated, and when it arrives just after, it is depressed, with the effect fading as the interval grows to a few tens of milliseconds.
Biological support is real but partial. Tim Bliss and Terje Lømo reported long-term potentiation in the rabbit hippocampus in 1973, a durable strengthening of synapses after high-frequency stimulation, and the NMDA receptor, which opens only when a presynaptic signal and postsynaptic depolarization coincide, is widely regarded as a molecular coincidence detector fitting Hebb's description. Yet the brain also uses homeostatic scaling, inhibitory plasticity, neuromodulators such as dopamine that carry reward information, and structural changes, so Hebb-style plasticity is one ingredient among several rather than a complete theory of learning. A common misconception is that Hebb proposed an algorithm for artificial networks or gave the equation above; he offered a qualitative psychological hypothesis, and the update rule is a later formalization.
The limits explain why Hebbian learning did not become the engine of deep learning. It is unsupervised and local: it finds correlations, not the correct answer, and has no mechanism for assigning credit through many layers, which is the job backpropagation does using a global error signal. Even so, the idea persists in self-organizing maps, in competitive learning, in Hopfield-style and modern dense associative memories, in neuromorphic chips and spiking-network research that implement STDP on hardware, and in biologically motivated work trying to approximate error-driven learning with local rules. Contrastive and predictive-coding schemes can be read as Hebbian updates modulated by error-like terms, suggesting that the principle survives as a building block even where the full recipe is different.
Read Hebbian learning as the idea that wiring follows activity: a single local rule, useful for memory and feature discovery, that explains how experience can be written into connections but not, on its own, how a system learns to be right.