Neural network pruning removes weights, channels, or entire structural units from a trained model to reduce its size and computational cost while preserving as much of its original accuracy as possible, exploiting the empirical observation that large trained networks are substantially over-parameterized relative to what is needed to represent the function they have learned. The result of pruning is sparsity: a model in which a large fraction of weights are exactly zero, either scattered arbitrarily through the weight tensors or concentrated into removable structural blocks, and the practical value of that sparsity depends entirely on whether the hardware and software running the model can convert removed weights into fewer FLOPs, less memory traffic, and lower latency rather than merely a smaller file on disk. This distinction between sparsity as a compression statistic and sparsity as a deployable speedup is the organizing tension of the entire field, because a pruning method that achieves striking weight-count reduction but no runtime benefit has not actually solved the problem practitioners care about.

Magnitude-based pruning ranks weights by absolute value and removes the smallest, resting on the heuristic that a weight close to zero contributes little to the network's output regardless of what the rest of the network is doing, and despite its simplicity this method remains a strong and frequently used baseline across model families. Global magnitude pruning ranks weights across the entire network, while layer-wise magnitude pruning enforces a target sparsity within each layer independently, and the choice matters because some layers are far more sensitive to weight removal than others — a global threshold can hollow out a sensitive early layer while barely touching an over-parameterized late layer, whereas a layer-wise threshold guarantees uniform sparsity at the cost of ignoring genuine differences in per-layer redundancy. Iterative magnitude pruning, which alternates between removing a small fraction of remaining weights and retraining (or fine-tuning) the survivors, generally reaches higher sparsity at a given accuracy target than one-shot pruning to the same final sparsity, because retraining lets the remaining weights compensate for what was removed at each step rather than absorbing the entire perturbation at once.

The lottery ticket hypothesis proposes that a dense, randomly initialized network contains a much smaller subnetwork which, if trained in isolation from that same initialization, can match the full network's accuracy, and this reframes pruning from a compression afterthought into a claim about what made the original training succeed in the first place. The standard procedure to find such a "winning ticket" trains the full network, prunes by magnitude, then resets the surviving weights to their original initial values (not their trained values) and retrains from that reset point; the finding that this reset-and-retrain procedure can match or exceed the pruned-and-fine-tuned result, for at least some architectures and sparsity levels, suggested that initialization — not merely the final trained values — carries meaningful information about which weights matter. This result has been influential but is not universal: whether a clean winning ticket exists, and how large the surviving subnetwork must be, depends heavily on architecture, dataset, and sparsity level, and larger or more heavily over-parameterized networks tend to yield tickets more reliably than smaller ones.

Effective sparsity is defined as the fraction of parameters set to zero, and this single number is frequently reported without the accompanying detail of granularity that determines whether it translates into any real-world benefit at all. For a network with $P$ total parameters of which $Z$ are exactly zero, effective sparsity is

and two models reported at the identical sparsity $s$ can have completely different deployment value depending on whether that zero pattern is unstructured (scattered, requiring specialized sparse kernels to exploit) or structured (concentrated into removable channels or blocks, exploitable by any dense-matrix hardware). Reporting $s$ alone, without specifying granularity and without measuring actual inference latency or memory bandwidth on target hardware, is therefore an incomplete and potentially misleading way to compare pruning methods.

Structured pruning removes entire channels, filters, attention heads, or other architecturally meaningful units rather than individual weights, and this structural constraint is what converts sparsity into an actual speedup on conventional dense hardware. Removing whole convolutional filters or transformer attention heads shrinks the weight tensor's dimensions directly, so the resulting network runs as an ordinary smaller dense model with no special sparse-matrix support required, whereas unstructured pruning leaves the tensor's nominal shape unchanged and merely sets a subset of its entries to zero, providing no speedup at all unless the runtime and hardware can skip those zeros efficiently. Structured pruning generally must remove more parameters than unstructured pruning to reach a comparable accuracy penalty, because it is a coarser, less selective form of removal — an entire channel is discarded even if most of its individual weights were still contributing something — but the resulting model requires no specialized inference infrastructure, which is why structured pruning dominates in deployment scenarios where the serving stack cannot exploit fine-grained sparsity.

Sensitivity- and gradient-based pruning criteria estimate the effect of removing a weight or structure on the training loss directly, rather than relying on magnitude as a proxy, and these methods generally identify a better set of removable parameters than magnitude alone at the cost of additional computation to estimate sensitivity. First-order methods approximate the loss change from removing a parameter using its gradient, while second-order methods incorporate curvature information (an approximation to the Hessian) to capture cases where a small-magnitude weight sits in a sharp region of the loss landscape and is actually important, or conversely where a larger-magnitude weight sits in a flat region and can be removed with little effect. These criteria matter more as target sparsity increases, because at low sparsity almost any reasonable criterion performs similarly, while at high sparsity — where the pruning decision genuinely trades off against accuracy — a criterion that better estimates true loss sensitivity can meaningfully outperform naive magnitude ranking.

Train the dense network to convergence, or start from a pretrained checkpoint → Select pruning granularity: unstructured, semi-structured, or structured → Choose a pruning criterion: magnitude, gradient-based sensitivity, or a structured-importance metric → Score all candidate weights or structures under the chosen criterion → Remove the lowest-scoring fraction according to the target sparsity for this step → Fine-tune or retrain the remaining network to recover accuracy lost in this step → Evaluate accuracy and effective sparsity against the target → Repeat prune-and-fine-tune iteratively if not yet at target sparsity, or stop if using one-shot pruning → Convert the pruned model into its deployment format: an ordinary smaller dense model for structured pruning, or a sparse format for unstructured pruning → Benchmark actual inference latency and memory footprint on target hardware, not just parameter count → Feed the achieved accuracy-versus-speedup trade-off back into the choice of granularity and target sparsity for future iterations

Pruning interacts with quantization and knowledge distillation as complementary rather than competing compression techniques, and production model compression pipelines typically combine multiple methods rather than relying on pruning alone. Quantization reduces the numerical precision of remaining weights and activations after pruning has reduced their count, so the two compound multiplicatively on model size and, with appropriate hardware support, on inference cost as well. Knowledge distillation trains a smaller or pruned student network to match a larger teacher's output distribution rather than only the original labels, which can recover accuracy that pruning alone would lose, particularly at higher sparsity levels where the pruned network's reduced capacity benefits from the richer training signal a teacher's soft targets provide. Because each technique addresses a different axis of model cost — parameter count, numerical precision, and effective capacity utilization — the state of the art in efficient model deployment generally applies pruning, quantization, and distillation together rather than treating pruning as a standalone solution.

Read neural network pruning through a granularity-versus-speedup lens: unstructured pruning can remove more parameters at a given accuracy cost, but that sparsity only becomes a real speedup on hardware built to exploit irregular zero patterns, while structured pruning removes fewer parameters yet turns directly into a smaller ordinary dense model that runs faster everywhere, and the right choice depends entirely on what the deployment hardware and software stack can actually do with the sparsity the pruning method produces.