roofline model performance analysis

**Roofline Model Performance Analysis** is **the visual performance modeling framework that characterizes the performance ceiling of a compute kernel as limited by either computational throughput or memory bandwidth — using arithmetic intensity (operations per byte transferred) as the key metric to identify the dominant bottleneck and guide optimization strategy**. **Roofline Model Fundamentals:** - **Arithmetic Intensity (AI)**: ratio of FLOPs to bytes transferred from/to memory — AI = total_FLOPs / total_bytes_moved; measured in FLOP/byte - **Performance Ceiling**: attainable performance = min(peak_FLOPS, peak_bandwidth × AI) — the lower of compute and memory bandwidth limits determines achievable performance - **Ridge Point**: the AI value where compute and memory ceilings intersect — kernels with AI below ridge point are memory-bound; above are compute-bound; ridge point = peak_FLOPS / peak_bandwidth - **Example**: GPU with 100 TFLOPS peak and 2 TB/s bandwidth has ridge point at 50 FLOP/byte — matrix multiply (AI ~100+) is compute-bound; vector addition (AI = 0.25) is memory-bound **Constructing the Roofline:** - **Memory Roof**: diagonal line with slope = peak memory bandwidth — applies to memory-bound kernels where performance scales linearly with arithmetic intensity - **Compute Roof**: horizontal line at peak computational throughput (FLOPS) — applies to compute-bound kernels where memory bandwidth is not the bottleneck - **Multiple Ceilings**: additional ceilings for L1/L2 cache bandwidth, special function unit throughput, and instruction-level parallelism — each ceiling creates a lower sub-roof that may limit specific kernels - **Achievable vs. Peak**: actual performance typically 50-80% of roofline ceiling — instruction overhead, pipeline stalls, and imperfect vectorization create gaps between achievable and theoretical performance **Using Roofline for Optimization:** - **Memory-Bound Kernels (AI < ridge point)**: optimization strategies focus on reducing data movement — caching/tiling, data compression, reducing precision (FP32→FP16), and eliminating redundant loads - **Compute-Bound Kernels (AI > ridge point)**: optimization strategies focus on increasing computational throughput — vectorization (SIMD/tensor cores), reducing instruction count, and increasing ILP - **Increasing AI**: algorithmic changes that increase FLOPs-per-byte-moved shift the kernel rightward on the roofline — tiling a matrix multiply to reuse cached data dramatically increases effective AI - **Profiling Integration**: NVIDIA Nsight Compute and Intel Advisor directly plot kernel performance against the roofline — shows how far each kernel is from the ceiling and which optimization would help most **The roofline model is the essential first-step analysis tool for performance optimization — it prevents the common mistake of optimizing compute throughput for a memory-bound kernel (which yields zero improvement) or vice versa, directing engineering effort to the actual bottleneck.**