gpu memory coalescing

**GPU Memory Coalescing** is the **hardware mechanism that combines multiple individual memory requests from threads within a warp (32 threads) into a single wide memory transaction — transforming 32 separate 4-byte reads into one 128-byte cache-line fetch when threads access consecutive addresses, which is the single most important optimization for achieving high memory bandwidth on GPUs**. **Why Coalescing Matters** GPU global memory (HBM or GDDR) delivers peak bandwidth only when accessed in large, aligned transactions (32-128 bytes). If each thread issues an independent random 4-byte read, the memory system must service 32 separate transactions per warp — consuming 32x the bus bandwidth for the same amount of useful data. With coalescing, the hardware detects that the 32 threads are accessing consecutive addresses and merges them into 1-4 aligned transactions. **Coalescing Rules** - **Fully Coalesced**: Thread i accesses address base + i * sizeof(element). All 32 threads' accesses fall within one or a few aligned 128-byte segments. Ideal — achieves near-peak bandwidth. - **Strided Access**: Thread i accesses base + i * stride. If stride > 1 element, threads' addresses spread across multiple cache lines. A stride of 2 wastes 50% of fetched data; a stride of 32 (column access in a row-major matrix) results in 32 separate transactions — the worst case. - **Random/Scattered**: Each thread accesses a random address. Every access is a separate transaction. Bandwidth utilization drops to 3-12% of peak. **Practical Optimization Patterns** - **Structure of Arrays (SoA) over Array of Structures (AoS)**: SoA layout ensures that consecutive threads accessing the same field read consecutive memory addresses. AoS causes strided access because consecutive threads skip over the other fields. - **Shared Memory Transpose**: Load a tile from global memory with coalesced access, store it in shared memory, then read from shared memory in any pattern (shared memory has no coalescing requirement since it uses banks, not wide transactions). - **Padding to Avoid Bank Conflicts**: When using shared memory as an intermediary, adding padding eliminates bank conflicts that would serialize access. **Hardware Evolution** Older GPUs (Fermi, Kepler) had strict alignment requirements for coalescing. Modern GPUs (Ampere, Hopper) have L1/L2 caches that partially mitigate uncoalesced access by caching fetched but unused bytes for subsequent requests from other warps. However, coalesced access still provides 5-10x better effective bandwidth than scattered access even on modern hardware. GPU Memory Coalescing is **the fundamental contract between the programmer and the hardware** — arrange your data so that neighboring threads access neighboring addresses, and the GPU rewards you with hundreds of GB/s of bandwidth; violate this contract, and performance collapses regardless of how many compute cores are available.