what is a systolic array
A systolic array is a grid-like arrangement of simple processing units that pass data to their immediate neighbors in a steady, rhythmic pattern, allowing an AI accelerator to perform the massive matrix multiplications neural networks require with remarkable efficiency — a core architectural idea behind chips like TPUs.
```flowchart
{
"rows": [
{ "type": "nodes", "items": [
{ "title": "Matrix multiplication needed at massive scale", "sub": "the core math behind neural network computation", "tone": "neutral" }
]},
{ "type": "arrow" },
{ "type": "group", "title": "Systolic array structure", "items": [
{ "title": "Grid of simple processing units", "sub": "each passes data to its immediate neighbors rhythmically", "tone": "green" }
]},
{ "type": "arrow" },
{ "type": "nodes", "items": [
{ "title": "Highly efficient matrix math", "sub": "data reused across the grid instead of repeatedly fetched from memory", "tone": "orange" }
]}
]
}
```
**A systolic array gets its name from how data flows through it in a steady, rhythmic pattern, much like blood pulsing through a body via a heartbeat.** Data enters the array from one side and passes from each processing unit to its immediate neighbor in a synchronized rhythm, with each unit performing a small piece of the overall calculation as data flows past it — this rhythmic, neighbor-to-neighbor data flow is where the "systolic" name comes from, drawing a direct analogy to the systolic rhythm of a heartbeat pushing blood through the body.
```svg
```
| Aspect | Traditional processor design | Systolic array |
|---|---|---|
| Data movement | Frequent trips back to memory | Data passed directly between neighboring units |
| Efficiency for matrix math | Lower — repeated memory access overhead | Higher — data reused as it flows through the grid |
| Flexibility | High — handles varied instructions | Narrower — optimized for structured, repetitive math |
| Common use | General-purpose computing | AI accelerators like TPUs |
**The systolic array's efficiency comes largely from reusing data as it flows through the grid, rather than repeatedly fetching it from memory.** In a systolic array, a piece of data entering the grid gets used by multiple processing units in sequence as it flows past them, rather than each unit independently fetching data from memory for every single calculation — this reuse dramatically reduces the number of memory accesses needed for a given amount of computation, directly addressing the memory-bandwidth bottleneck that often limits AI accelerator performance.
**Systolic arrays trade general-purpose flexibility for extreme efficiency at one specific, highly structured type of computation.** The simple processing units in a systolic array are designed around a very specific, repetitive computational pattern — passing and accumulating values in a fixed rhythm — rather than being able to execute arbitrary instructions like a general-purpose processor core; this narrow specialization is precisely what allows a systolic array to achieve such high efficiency specifically for matrix multiplication, at the cost of being far less flexible for other types of computation.
**The systolic array concept predates modern AI hardware by decades, originally developed for other specialized computing applications before finding its most prominent modern use in AI accelerators.** The underlying idea of passing data rhythmically between simple processing units in a grid was explored in computer architecture research long before neural networks became a major computing workload — it was the rise of AI's massive matrix multiplication demands that made the systolic array's specific strengths newly, enormously valuable for a mainstream and rapidly growing computing application.
Read the systolic array through a rhythmic-relay lens: rather than each processing unit independently fetching and processing data on its own, a systolic array has data flow rhythmically from unit to unit across a grid, each one contributing a small piece of the calculation as the data passes by — an elegantly specialized design for exactly the kind of massive, repetitive matrix math that modern AI accelerators need to perform.