Dataflow Architecture and Programming is the computation model where operations execute as soon as all their input data becomes available, rather than following a sequential program counter — naturally expressing parallelism through data dependency graphs where independent operations fire concurrently without explicit thread management, used in hardware (systolic arrays, FPGAs), software frameworks (TensorFlow graphs, Apache Flink), and modern ML compilers that analyze dataflow to maximize pipeline and instruction-level parallelism.

Dataflow vs. Control Flow

Dataflow Graph Execution

  [Read A]  [Read B]  [Read C]     ← All can fire immediately
      \      /    \      /
     [A + B]      [B * C]          ← Fire when inputs ready
         \          /
         [Result + Product]        ← Fire when both done
              |
          [Write Output]           ← Fire when input ready

• Nodes = operations. Edges = data dependencies.
• Node fires when ALL input tokens (data values) are available.
• Independent nodes fire simultaneously → automatic parallelism.

Static vs. Dynamic Dataflow

Software Dataflow Frameworks

Hardware Dataflow

• Systolic arrays (TPU): Data flows through PE (processing element) array → each PE fires when data arrives from neighbor.
• FPGA: Naturally dataflow → operations wired together, data flows through pipeline.
• CGRA (Coarse-Grained Reconfigurable Array): Programmable dataflow fabric.
• Cerebras WSE: Dataflow between cores on wafer-scale chip.

ML Compiler Dataflow Analysis

# XLA / TVM / Triton analyze dataflow to optimize:
# 1. Operator fusion: Merge connected nodes → one kernel
# 2. Memory allocation: Reuse buffers when producer-consumer lifetimes don't overlap
# 3. Scheduling: Topological sort of graph → maximize parallelism
# 4. Pipelining: Stream data through fused operators

# Example: y = relu(matmul(x, W) + b)
# Dataflow: x,W → matmul → +b → relu → y
# Fused: Single kernel (matmul → add → relu) with no intermediate materialization

Stream Processing as Dataflow

 [Kafka Source] → [Parse JSON] → [Filter] → [Aggregate] → [Sink]
                      ↑              ↑           ↑
                 All stages run continuously in parallel
                 Data flows through pipeline as it arrives

• Apache Flink: Dataflow graph with backpressure → automatically balances throughput.
• Throughput: Limited by slowest stage (pipeline parallelism).

Dataflow programming is the natural expression of parallelism that eliminates explicit synchronization — by modeling computation as data flowing through a graph of operations, dataflow makes parallelism implicit in the structure of the computation itself, which is why it forms the foundation of ML compiler optimizations, FPGA designs, stream processing systems, and the increasingly graph-based execution models of modern AI frameworks.