GPU Memory Coalescing is the hardware optimization where adjacent threads in a warp (32 threads) that access adjacent memory addresses have their individual memory requests combined into a single wide memory transaction (32, 64, or 128 bytes) — reducing the number of memory transactions by up to 32x and achieving peak memory bandwidth, while uncoalesced access patterns (strided, random) generate separate transactions per thread, reducing effective bandwidth to 3-10% of peak.

How Coalescing Works

When a warp executes a load instruction, the memory controller examines all 32 threads' addresses:

• Fully Coalesced: Thread i accesses address BASE + i×sizeof(element). All 32 addresses fall within a single 128-byte cache line. The memory controller issues one 128-byte transaction. Full bandwidth.
• Partially Coalesced: Addresses span 2-4 cache lines. 2-4 transactions issued. 50-25% of peak bandwidth.
• Fully Uncoalesced: Each thread accesses a different cache line. 32 separate transactions. 3% of peak bandwidth. Performance disaster.

Access Patterns and Their Coalescing Behavior

• Stride-1 (Contiguous): Thread i reads array[i]. Perfectly coalesced. Full bandwidth.
• Stride-N: Thread i reads array[i×N]. If N=32, each thread hits a different sector of the cache — completely uncoalesced. Common when accessing a column of a row-major 2D array.
• Random (Scatter/Gather): Thread i reads array[index[i]] where index is data-dependent. Typically fully uncoalesced. Each thread may hit a different cache line.

Array of Structures vs. Structure of Arrays

The most impactful data layout decision for GPU performance:

// AoS (Array of Structures) — BAD for GPU
struct Particle { float x, y, z, mass; };
Particle particles[N];
// Thread i reads particles[i].x → stride-4 access (every 16 bytes)

// SoA (Structure of Arrays) — GOOD for GPU  
float x[N], y[N], z[N], mass[N];
// Thread i reads x[i] → stride-1 access (perfectly coalesced)

Converting AoS to SoA is often the single highest-impact GPU optimization — can improve memory-bound kernel performance by 4-8x.

L1/L2 Cache Interaction

Modern GPUs (Ampere, Hopper) have configurable L1 caches (up to 228 KB per SM on H100). Uncoalesced accesses that hit L1 cache are less penalized than L1 misses. For random access patterns, increasing L1 cache size (at the expense of shared memory) can partially mitigate uncoalesced access.

Alignment Requirements

Aligned loads (address divisible by transaction size) avoid split transactions. Built-in vector types (float4, int4) guarantee 16-byte aligned loads. __align__ directive in CUDA forces alignment of arrays and structures. Misaligned base addresses can cause every warp to generate two transactions instead of one.

Memory Coalescing is the single most important GPU performance rule — determining whether a memory-bound kernel achieves 80-100% of peak bandwidth or limps along at 3-10%, making data layout design the first and most impactful optimization decision in GPU programming.