roofline model

**Roofline Model** — a visual framework for understanding whether a computation is limited by compute throughput or memory bandwidth, guiding optimization efforts. **The Model** $$Performance = min(Peak\_FLOPS, Peak\_BW \times OI)$$ Where: - **OI (Operational Intensity)** = FLOPs / Bytes transferred from memory - **Peak FLOPS**: Maximum compute throughput (e.g., 10 TFLOPS) - **Peak BW**: Maximum memory bandwidth (e.g., 900 GB/s for HBM) **Two Regimes** - **Memory-Bound** (low OI): Performance limited by how fast data can be fed to compute units. Most deep learning inference, sparse computations - **Compute-Bound** (high OI): Performance limited by arithmetic throughput. Dense matrix multiply, convolutions with large batch sizes **Example (NVIDIA A100)** - Peak: 19.5 TFLOPS (FP32), 2 TB/s (HBM2e) - Ridge point: 19.5T / 2T = ~10 FLOP/Byte - If your kernel does < 10 FLOP per byte loaded → memory-bound - If > 10 → compute-bound **Optimization Strategy** - Memory-bound → reduce data movement (tiling, caching, compression, data reuse) - Compute-bound → use tensor cores, vectorization, reduce wasted compute **The roofline model** quickly tells you what's limiting performance and where to focus optimization — essential for HPC and GPU programming.