roofline model performance
**The Roofline Model** is the **visual performance analysis framework that plots achievable computation throughput (FLOPS) against arithmetic intensity (FLOPS/byte) — creating a "roofline" ceiling defined by peak compute capacity (horizontal) and peak memory bandwidth (diagonal slope) that immediately reveals whether a kernel is compute-bound or memory-bound and quantifies the gap between achieved and theoretically achievable performance**.
**The Model**
For a given hardware platform:
- **Peak Compute (P)**: Maximum floating-point operations per second (e.g., 100 TFLOPS for an NVIDIA A100 at FP32).
- **Peak Memory Bandwidth (B)**: Maximum bytes per second from main memory (e.g., 2 TB/s for HBM2e).
- **Arithmetic Intensity (AI)**: FLOPS performed per byte loaded from memory for a specific kernel. AI = Total FLOPS / Total Bytes Transferred.
The roofline ceiling for a kernel with arithmetic intensity AI is: Achievable FLOPS = min(P, B × AI).
- If B × AI < P: the kernel is **memory-bound** — performance is limited by how fast data arrives, not how fast the ALUs compute. The kernel rides the diagonal (bandwidth-limited) slope.
- If B × AI ≥ P: the kernel is **compute-bound** — the ALUs are the bottleneck, and the kernel hits the horizontal (compute) ceiling.
**Reading the Roofline Plot**
```
Performance | _______________ (Peak Compute)
(GFLOPS) | /
| / (Bandwidth Ceiling)
| /
| / * Kernel A (memory-bound, 70% of roof)
| /
| / * Kernel B (compute-bound, 45% of roof)
| /
|/______________________________
Arithmetic Intensity (FLOP/Byte)
```
**Kernel A** is memory-bound at 70% of the bandwidth roof — optimizing should focus on data reuse (tiling, caching) to increase AI or reducing unnecessary loads.
**Kernel B** is compute-bound at 45% of the compute roof — optimizing should focus on vectorization, ILP, and instruction mix.
**Extended Roofline**
The basic model can be extended with additional ceilings:
- **L1/L2 Cache Bandwidth**: Separate diagonal ceilings for each cache level, showing whether a kernel is bound by main memory, L2, or L1 bandwidth.
- **Mixed Precision**: Different horizontal ceilings for FP64, FP32, FP16, INT8 — reflecting the different peak throughputs of each data type.
- **Special Function**: Separate ceilings for transcendental functions (sin, exp) which have lower throughput than FMA operations.
**Practical Application**
- GEMM (matrix multiply) has AI = O(N) — deep in the compute-bound region. Achieved performance should approach 90%+ of peak FLOPS.
- SpMV (sparse matrix-vector multiply) has AI = O(1) — firmly memory-bound. Performance is limited to 5-10% of peak FLOPS regardless of optimization.
- Convolution AI depends on filter size, channel count, and batch size — can be either compute-bound or memory-bound depending on configuration.
The Roofline Model is **the performance engineer's X-ray machine** — instantly diagnosing whether a kernel is starved for data or saturated with computation, and quantifying exactly how much performance headroom remains before hitting the hardware's fundamental limits.