Roofline Model — a visual framework for understanding whether a computation is limited by compute throughput or memory bandwidth, guiding optimization efforts.

The Model

Where:

• OI (Operational Intensity) = FLOPs / Bytes transferred from memory
• Peak FLOPS: Maximum compute throughput (e.g., 10 TFLOPS)
• Peak BW: Maximum memory bandwidth (e.g., 900 GB/s for HBM)

Two Regimes

• Memory-Bound (low OI): Performance limited by how fast data can be fed to compute units. Most deep learning inference, sparse computations
• Compute-Bound (high OI): Performance limited by arithmetic throughput. Dense matrix multiply, convolutions with large batch sizes

Example (NVIDIA A100)

• Peak: 19.5 TFLOPS (FP32), 2 TB/s (HBM2e)
• Ridge point: 19.5T / 2T = ~10 FLOP/Byte
• If your kernel does < 10 FLOP per byte loaded → memory-bound
• If > 10 → compute-bound

Optimization Strategy

• Memory-bound → reduce data movement (tiling, caching, compression, data reuse)
• Compute-bound → use tensor cores, vectorization, reduce wasted compute

The roofline model quickly tells you what's limiting performance and where to focus optimization — essential for HPC and GPU programming.