Stride

Stride is the number of positions a convolution or pooling window moves between one application and the next as it slides across its input. With a stride of 1, the window visits every position and the output is nearly as large as the input; with a stride of 2, it visits every second position in each spatial direction, so the output is about half as wide and half as tall. Stride is therefore the main built-in dial for downsampling in a convolutional network, controlling how quickly spatial resolution is traded for computation and for a broader view of the image.

The output size follows from one formula, and working it through once makes network diagrams readable. For an input of width W, kernel size K, padding P on each side, and stride S, the output width is floor((W − K + 2P) / S) + 1, where floor rounds down. AlexNet's first layer (Krizhevsky, Sutskever, and Hinton, 2012) applies 11×11 kernels with stride 4 to a 227-pixel-wide input with no padding, giving (227 − 11) / 4 + 1 = 55. The first layer of ResNet-50 uses a 7×7 kernel with padding 3 and stride 2 on a 224-pixel input, giving floor((224 − 7 + 6) / 2) + 1 = floor(111.5) + 1 = 112; a following 3×3 max-pooling layer with stride 2 and padding 1 gives floor((112 − 3 + 2) / 2) + 1 = 56. A 224-pixel image has fallen to 56 pixels after two steps, a sixteen-fold reduction in the number of spatial positions every later layer must process.

Stride interacts with kernel size to decide whether the windows overlap, and that is where its most common misconception lives. When the stride is smaller than the kernel, as in a 3×3 kernel with stride 2, neighboring windows overlap and every input pixel still influences the output, so a stride-2 convolution does not simply throw away half the image. When the stride equals the kernel size the windows tile the input exactly with no overlap, and when the stride exceeds the kernel size some pixels are never seen at all, which is almost always a bug. The Vision Transformer (Dosovitskiy and colleagues, 2020) exploits the exact-tiling case: its patch embedding is equivalent to a convolution with a 16×16 kernel and stride 16, cutting the image into non-overlapping patches and projecting each one into a token.

Strided convolutions can replace pooling layers, a point demonstrated by Springenberg, Dosovitskiy, Brox, and Riedmiller in 2014. Their "Striving for Simplicity: The All Convolutional Net" showed that max pooling could be dropped in favor of a convolution with stride 2 without hurting accuracy on benchmarks such as CIFAR-10, because the network can learn its own downsampling instead of using a fixed maximum or average. Modern architectures follow both habits: ResNets downsample by giving the first convolution of a stage a stride of 2, and many efficient or generative models rely on strided convolutions alone. The cost is the loss of the built-in tolerance that pooling offered, a problem taken up in the sampling-theory view below.

Aliasing is the underappreciated hazard of large strides. Sampling every second position of a signal that has not been smoothed first violates the sampling theorem, so small shifts of the input can change the output sharply; Zhang's 2019 paper "Making Convolutional Networks Shift-Invariant Again" documented this and proposed blurring before subsampling (anti-aliased pooling) as a remedy. The mirror image of downsampling is upsampling, where a transposed convolution with stride 2 doubles resolution in decoders, generators, and segmentation networks; if the kernel size is not divisible by the stride, uneven overlap creates the checkerboard artifacts analyzed by Odena, Dumoulin, and Olah in 2016, which is why resize-then-convolve is often preferred.

In practice, stride is chosen jointly with padding and depth so that the sizes divide cleanly. An input whose width is not compatible with the formula loses its last rows or columns to the floor, which can quietly misalign features with a skip connection in an encoder-decoder or shift object detection boxes by a few pixels. Stride also differs from dilation, which spreads the kernel's taps apart rather than moving the window faster, and from padding, which controls the border instead of the step. Frameworks print these sizes layer by layer, and checking them against the formula is the fastest way to debug a shape error.

Read stride as the step size of the sliding window and therefore the rate of spatial compression: each unit of stride buys speed and a wider receptive view at the price of resolution, and the output-size formula is how to know in advance exactly what that purchase costs.