dataflow

Dataflow is an execution model in which a computation is expressed as a graph of operations, and each operation runs as soon as its input data is available, rather than in a fixed order dictated by a program counter. It is the organizing idea behind most modern AI accelerators, and it is the reason a systolic array or a spatial NPU can keep thousands of arithmetic units busy where a conventional processor would stall.\n\n```svg\n\n```\n\n**The contrast is with the von Neumann model.** A conventional CPU is *control-driven*: a program counter walks through instructions one at a time, each fetching operands from a central memory across a shared bus — the classic von Neumann bottleneck. A dataflow machine is *data-driven*: there is no program counter, and an operator fires the moment its operands (its "tokens") arrive on its input edges. Order is implied by the data dependencies in the graph, not by an instruction sequence, so everything that is independent can run at once.\n\n**Neural networks are already dataflow graphs.** A model is a directed acyclic graph of tensor operations — matmuls, convolutions, activations — with edges that are data dependencies. Mapping that graph directly onto hardware, so each operator has its own processing elements and passes results straight to the next operator, removes the round-trips to central memory that dominate energy and latency. This producer-to-consumer passing is why spatial architectures are so efficient on dense tensor pipelines.\n\n**In AI accelerators the word has a second, more specific meaning.** Beyond "data-driven execution," *dataflow* is the term for how a loop nest schedules reuse — which operand you keep stationary inside each processing element while others stream past. This choice determines how many times a value read from memory gets reused, and reuse is where the energy-per-operation advantage comes from. The well-known taxonomy from the Eyeriss work names the common patterns.\n\n**Weight-stationary, output-stationary, row-stationary.** In a weight-stationary dataflow each PE holds a weight and streams activations past it, reusing that weight across many multiply-accumulates — this is the systolic array of a TPU. In an output-stationary dataflow the partial sum stays resident and accumulates in place. Row-stationary, used in Eyeriss, keeps a row of a convolution local to maximize reuse of both weights and activations at once. Each pattern trades which data moves against which stays put.\n\n**Tokens and asynchrony replace global synchronization.** Because operators fire on operand availability, a dataflow fabric coordinates through local producer-consumer handshakes — dataflow tokens — instead of a global clock-step barrier. That asynchrony is what lets pipeline stages and independent branches overlap without central scheduling overhead, and it is why tiling and keeping working sets in local memory matter so much: the whole model is to move data as short a distance as possible.\n\n| Property | von Neumann | Dataflow |\n|---|---|---|\n| What triggers work | program counter (control) | operand availability (data) |\n| Main bottleneck | memory–ALU bus | edge bandwidth and graph mapping |\n| Parallelism | limited, sequential | natural, across the whole graph |\n| Best fit | control-heavy general code | dense tensor pipelines |\n\n| Reuse dataflow | Kept stationary in PE | What it reuses | Seen in |\n|---|---|---|---|\n| Weight-stationary | weights | a weight across many activations | systolic arrays, TPU |\n| Output-stationary | partial sums | accumulation of one output | many GEMM engines |\n| Row-stationary | a conv row | weights and activations together | Eyeriss |\n| No local reuse | nothing | streams every operand | simple SIMD |\n\nRead dataflow through an *operand-reuse-and-firing* lens rather than an *instruction* lens: the model's whole advantage is that work is triggered by data arriving instead of by a program counter, and that operations pass results directly to one another instead of through central memory. Whether the word means "data-driven execution" or "which operand stays stationary in each PE," the underlying question is the same — how to keep arithmetic units fed by moving data the shortest possible distance, which is exactly the problem every AI accelerator is built to solve.\n