roofline model analysis
**Roofline Model Analysis** is the **visual performance modeling framework that plots achievable performance (FLOP/s) against arithmetic intensity (FLOP/byte) to determine whether a computation is memory-bound or compute-bound** — providing immediate insight into the performance bottleneck and the maximum achievable speedup, making it the most practical first-step analysis tool for understanding and optimizing the performance of any computational kernel on any hardware.
**Roofline Construction**
- **X-axis**: Arithmetic Intensity (AI) = FLOPs / Bytes transferred (operational intensity).
- **Y-axis**: Attainable Performance (GFLOP/s or TFLOP/s).
- **Memory ceiling**: Diagonal line with slope = memory bandwidth. Performance = AI × BW.
- **Compute ceiling**: Horizontal line at peak compute rate.
- **Performance** = min(Peak_Compute, AI × Peak_Bandwidth).
**Roofline for NVIDIA A100**
```
Peak FP32: 19.5 TFLOPS
HBM Bandwidth: 2.0 TB/s
Ridge Point: 19,500 / 2,000 = 9.75 FLOP/byte
TFLOP/s
19.5 |__________________________ (compute ceiling)
| /
| /
| / ← memory ceiling (slope = 2 TB/s)
| /
| /
| /
| /
| /
| /
|/__________________________ AI (FLOP/byte)
9.75
(ridge point)
```
- **Left of ridge**: Memory-bound → optimize memory access (coalescing, caching, reuse).
- **Right of ridge**: Compute-bound → optimize computation (SIMD, FMA, algorithm efficiency).
**Computing Arithmetic Intensity**
| Kernel | FLOPs/element | Bytes/element | AI | Bound |
|--------|-------------|-------------|-----|-------|
| Vector add (a+b→c) | 1 | 12 (3×4B) | 0.08 | Memory |
| Dot product | 2N | 8N+4 | ~0.25 | Memory |
| Dense GEMM (NxN) | 2N³ | 3×4N² | N/6 | Compute (for large N) |
| 1D stencil (3-point) | 2 | 4 (with reuse) | 0.5 | Memory |
| SpMV (sparse) | 2×NNZ | 12×NNZ | 0.17 | Memory |
**Roofline Extensions**
| Ceiling | Description |
|---------|------------|
| L1 bandwidth ceiling | Performance bound by L1 cache bandwidth |
| L2 bandwidth ceiling | Performance bound by L2 cache bandwidth |
| SIMD ceiling | Penalty for non-vectorized code |
| FMA ceiling | Penalty for not using fused multiply-add |
| Tensor Core ceiling | Peak when using tensor cores (mixed precision) |
**Using Roofline for Optimization**
1. **Profile kernel**: Measure actual FLOP/s and bytes transferred.
2. **Plot on roofline**: Where does the kernel sit relative to ceilings?
3. **If below memory ceiling**: Memory access inefficiency → fix coalescing, add caching.
4. **If at memory ceiling**: Memory-bound → increase AI (algorithm change, tiling, reuse).
5. **If at compute ceiling**: Compute-bound → use wider SIMD, tensor cores, better algorithm.
**Tools**
- **Intel Advisor**: Automated roofline analysis for CPU.
- **NVIDIA Nsight Compute**: Roofline chart for GPU kernels.
- **Empirical Roofline Toolkit (ERT)**: Measures actual machine ceilings.
The roofline model is **the most effective framework for understanding computational performance** — by instantly revealing whether a kernel is memory-bound or compute-bound and quantifying the gap to peak performance, it guides optimization effort toward the actual bottleneck rather than wasting time on non-limiting factors.