A systolic array is a grid of multiply-accumulate (MAC) cells that pumps data rhythmically from each cell to its neighbor, reusing every operand many times as it flows through — which is how a TPU or an NPU multiplies matrices at very high efficiency.\n\nThe matrix multiply is the workload. Neural networks are dominated by matrix multiplications: activations times weights, layer after layer. A systolic array maps that operation directly onto silicon — one MAC unit per grid cell — instead of shuttling every operand back and forth to a register file.\n\nData moves like a heartbeat. In the common weight-stationary scheme each cell holds one weight; activations stream in from the left and march right one cell per clock, while partial sums accumulate downward. Every value that enters is used by an entire row or column before it leaves, so an N x N array performs N^2 MACs per cycle while reading each operand from memory only once. That reuse — not raw clock speed — is the source of the efficiency.\n\n| Property | Systolic array | Conventional CPU/GPU lane |\n|---|---|---|\n| Compute per cycle | N^2 MACs (N x N grid) | a few MACs per core |\n| Operand reuse | each value feeds a whole row/col | reload from register/cache |\n| Data motion | local, neighbor-to-neighbor | global, via register file |\n| Best at | dense matmul / convolution | branchy, irregular code |\n| Example | Google TPU MXU (256 x 256) | general-purpose core |\n\n``svg\n\n``\n\nIt trades flexibility for density. A systolic array is spectacular at dense linear algebra and mediocre at everything else — irregular, branchy, or sparse work maps poorly onto the lockstep grid. That is why it shows up as a dedicated block (the TPU's MXU, Nvidia's tensor cores, NPU matrix engines) sitting alongside general-purpose cores rather than replacing them.\n\nRead the systolic array through a quant lens rather than a hardware lens: its whole advantage is arithmetic intensity — MACs performed per byte fetched from memory. An N x N array raises operand reuse to order N, which pushes a matmul off the memory-bound side of the roofline and onto the compute-bound side, where the array's peak MACs per cycle — not HBM bandwidth — sets the throughput.