l2 cache

**L2 cache** is the **shared on-chip cache level that serves all streaming multiprocessors before traffic reaches high-latency HBM** - it acts as a global reuse and coherence layer for data exchanged across blocks and kernels on the same GPU. **What Is L2 cache?** - **Definition**: Unified cache between per-SM caches and off-chip memory, visible to all compute units. - **Role**: Absorbs repeated global-memory accesses and reduces expensive HBM transactions. - **Coherence Point**: Writes from one SM can be observed by others through L2-backed memory consistency behavior. - **Capacity Context**: Larger than L1 or shared memory but slower than those nearest compute tiers. **Why L2 cache Matters** - **Bandwidth Relief**: High L2 hit rate reduces pressure on external memory channels. - **Cross-SM Reuse**: Common tensors accessed by many blocks can be served with lower latency. - **Kernel Throughput**: Memory-heavy kernels often scale with effective L2 behavior. - **Energy Efficiency**: On-chip reuse consumes less energy than repeated off-chip fetches. - **System Balance**: Optimized L2 utilization improves overall compute to memory balance. **How It Is Used in Practice** - **Access Locality**: Design kernels so nearby threads and successive blocks touch overlapping address regions. - **Working-Set Tuning**: Adjust tile size and launch strategy to fit hot data within L2 residency windows. - **Profiling**: Track L2 hit rate and throughput counters to identify memory hierarchy bottlenecks. L2 cache is **the shared memory-traffic stabilizer for GPU-wide execution** - strong L2 locality can significantly raise end-to-end kernel performance.