Global Memory in GPU architecture refers to the main off-chip DRAM accessible by all threads across all streaming multiprocessors (SMs).

What Is Global Memory?

• Capacity: 4GB to 80GB+ on modern GPUs (HBM2/GDDR6)
• Bandwidth: 500GB/s to 3TB/s depending on memory type
• Latency: 400-800 clock cycles (much slower than shared memory)
• Scope: Accessible by all threads in all blocks

Why Global Memory Matters

Global memory is where large datasets, model weights, and results reside. Despite high bandwidth, poor access patterns cause performance bottlenecks.

// Global memory access example
__global__ void kernel(float *globalData) {
    int idx = blockIdx.x * blockDim.x + threadIdx.x;
    
    // Coalesced access - threads read consecutive addresses
    float val = globalData[idx];  // Good pattern
    
    // Strided access - inefficient, multiple transactions
    float val2 = globalData[idx * 32];  // Bad pattern
}

Optimization Tips:

• Coalesce memory accesses (consecutive threads → consecutive addresses)
• Use shared memory as cache for repeated accesses
• Align data structures to 128-byte boundaries