cnn

A convolutional neural network is the architecture that assumes its input is a grid — an image, a spectrogram, a volume — and that the useful patterns in that grid are *local* and appear the *same* wherever they occur. Those two assumptions, locality and translation invariance, are not incidental; they are the whole point. Baking them into the wiring is what let CNNs dominate computer vision for a decade with a fraction of the parameters a fully-connected network would need. The way to understand a CNN is not as a stack of mysterious layers but as those two assumptions made mechanical.\n\n**The convolution is a small filter slid across the whole image, reusing the same weights at every position.** Instead of connecting every pixel to every neuron, a convolutional layer learns a tiny bank of filters — say 3x3 weight patches — and drags each one across the image, computing a dot product at every location to produce a *feature map* that lights up wherever that filter's pattern appears. This is *weight sharing*: the same few weights are reused everywhere, which slashes the parameter count and hard-codes the assumption that a vertical edge is a vertical edge whether it sits in the top-left corner or dead center. A filter only ever sees a small *receptive field* of nearby pixels, encoding the assumption that what matters is local.\n\n**Pooling and stride shrink the spatial map, building a hierarchy from edges to objects.** Between convolutions, a CNN downsamples — max-pooling keeps the strongest response in each little neighborhood, or a strided convolution steps across more coarsely — so deeper layers see a wider swath of the original image through the same small filter. Stack these and the receptive field grows layer by layer: the first layers fire on edges and color blobs, the middle layers assemble those into textures and parts, and the deepest layers respond to whole objects. This coarse-to-fine hierarchy is the second reason CNNs match natural images so well, since real-world visual structure is itself compositional.\n\n**The classic lineage is a story of going deeper once the tricks to train depth arrived.** LeNet proved the idea on digits in the 1990s; AlexNet blew open ImageNet in 2012 with ReLUs, dropout, and GPUs; VGG showed that stacks of small 3x3 filters were enough; and ResNet's residual connections finally made networks hundreds of layers deep trainable by letting gradients skip. That ResNet trick is the single most important enabler of the deep-learning era and reaches far beyond vision. The CNN's reign ended only when Vision Transformers showed that, given enough data, self-attention could learn the same spatial structure without hard-coding it — trading the CNN's built-in bias for raw scale.\n\n| Component | What it does | Assumption it encodes |\n|---|---|---|\n| Convolution filter | Slides shared weights over the grid | Features are local |\n| Weight sharing | Same filter reused everywhere | Translation invariance |\n| Receptive field | Each unit sees a local patch | Nearby pixels relate |\n| Pooling / stride | Downsamples the spatial map | Structure is compositional |\n| Depth (LeNet -> ResNet) | Stacks layers into a hierarchy | Objects are parts of parts |\n\n```svg\n\n```\n\nThe unhelpful way to learn a CNN is as a recipe: convolution, ReLU, pool, repeat, flatten, classify. That checklist misses why the recipe is shaped the way it is. Every ingredient exists to serve one of two assumptions — that useful features are local, and that they mean the same thing anywhere in the frame — and the whole architecture is those assumptions turned into hardware. Read a CNN through a what-structure-it-assumes-about-the-data lens rather than a which-layers-to-stack lens, and its strengths and limits both fall out immediately: it is unbeatable when the data really is a grid of locally-related, position-independent features, and it struggles precisely when the data is not, which is exactly the gap that attention stepped in to fill.