Pruning removes the parts of a trained neural network that contribute least, and sparsity is the result: a model in which most weights are zero. The premise is that large networks are heavily over-parameterized — they have far more weights than they strictly need — so a large fraction can be deleted with little or no loss in accuracy. Pruning is a core model-compression technique for shrinking memory footprint, cutting energy use, and speeding up inference, especially on edge and cost-sensitive deployments, and it composes with quantization and distillation.\n\n``svg\n\n``\n\nThe first choice is unstructured versus structured. Unstructured pruning zeros out individual weights, usually the ones with the smallest magnitude; it reaches very high sparsity with excellent accuracy retention, but the surviving pattern is irregular, so a dense GPU sees no speedup without specialized sparse kernels. Structured pruning instead removes whole units — channels, filters, or attention heads — producing a smaller dense model that runs faster on any hardware, at the cost of somewhat lower achievable sparsity and a bigger accuracy hit per weight removed.\n\nThe standard recipe is prune, then recover, repeatedly. You rank weights by an importance score — magnitude is the simplest, but gradient-, Taylor-, and Fisher-based scores estimate impact more carefully — remove the least important, then fine-tune the network to recover the accuracy lost. Doing this gradually over several rounds (iterative pruning) reliably beats removing everything in a single pass (one-shot pruning), because the network gets a chance to reallocate capacity between cuts.\n\nThe Lottery Ticket Hypothesis reframed what pruning finds. Frankle and Carbin showed that a dense network contains a sparse "winning subnetwork" that, when trained from the original initialization, can match the full network's accuracy. This shifted the mental model from "compress a trained model" toward "a trainable sparse subnetwork was hiding inside all along," and it spurred a wave of research into finding such subnetworks early rather than after full training.\n\nTurning sparsity into real speed is a hardware problem. A model can be ninety percent zeros and still run at full dense speed, because general matrix hardware processes the zeros anyway. Getting wall-clock gains requires patterns the hardware can exploit: structured pruning that yields a genuinely smaller dense model, or semi-structured "N:M" sparsity — such as NVIDIA's 2:4, where two of every four weights are zero — which maps directly onto sparse tensor cores. This is why deployment-focused work favors structured and N:M patterns over free-form unstructured sparsity.\n\nThe payoff and the caveats. Pruning can substantially cut model size and energy while preserving most accuracy, and it stacks with other compression methods for large combined gains. The caveats are that accuracy degrades as sparsity climbs toward extreme levels, the prune-and-fine-tune loop adds training cost, and the theoretical reduction in floating-point operations often exceeds the actual speedup once memory layout and hardware realities are accounted for.\n\n| Type | What it removes | Achievable sparsity | Where it speeds up |\n|---|---|---|---|\n| Unstructured (magnitude) | individual weights | very high | only with sparse kernels/hardware |\n| Structured | channels, filters, heads | moderate | any hardware (smaller dense model) |\n| Semi-structured N:M (2:4) | a fixed pattern per block | around one half | sparse tensor cores |\n| Lottery ticket | finds a winning subnetwork | high | an insight about initialization |\n\nRead pruning through a what-can-the-hardware-exploit lens rather than a how-many-weights-can-I-delete lens: reaching high sparsity is the easy part, but the removed weights only become real speed when the surviving pattern is structured or N:M regular — which is why the practical art is trading a little sparsity for a layout the chip can actually run faster.\n