GPU Compute Shaders are the programmable pipeline stages that execute general-purpose parallel computations on GPU hardware outside the traditional graphics rendering pipeline — enabling thousands of threads to process data in parallel using the GPU's massive SIMD architecture for workloads ranging from physics simulation and image processing to machine learning inference and cryptographic operations.

From Graphics to General Compute

GPUs were originally fixed-function graphics pipelines. The introduction of programmable shaders (vertex, fragment) revealed that the underlying hardware — thousands of ALUs with high-bandwidth memory — was a powerful general-purpose parallel processor. Compute shaders (introduced in OpenGL 4.3, DirectX 11, Vulkan 1.0) formalized this by providing a non-graphics entry point to GPU hardware.

Execution Model

• Workgroup (Thread Block): The programmer dispatches a grid of workgroups. Each workgroup contains a fixed number of threads (e.g., 256) that execute the same shader program (SIMT model). Threads within a workgroup can communicate through shared memory and synchronize with barriers.
• Dispatch: The CPU issues a dispatch command specifying the grid dimensions (e.g., 128×128×1 workgroups). The GPU scheduler distributes workgroups across available Compute Units (CUs) / Streaming Multiprocessors (SMs).
• SIMD Execution: Within each CU/SM, threads are grouped into wavefronts (AMD, 64 threads) or warps (NVIDIA, 32 threads) that execute the same instruction in lockstep. Divergent branches cause serialization within the wavefront/warp.

Memory Hierarchy

Compute Shader Use Cases

• Image Processing: Convolutions, tone mapping, histogram computation — each pixel maps to one thread, processing the entire image in a single dispatch.
• Physics Simulation: Particle systems, fluid dynamics (SPH), cloth simulation — each particle/cell is a thread, neighbor interactions use shared memory.
• ML Inference: Matrix multiplications (GEMM) for neural network layers — workgroups tile the output matrix, using shared memory to cache input tiles for reuse.
• Prefix Sum / Reduction: Fundamental parallel primitives that map naturally to the workgroup→barrier→workgroup execution pattern.

Performance Optimization

• Occupancy: Keep enough wavefronts/warps in-flight to hide memory latency. Limited by register usage, shared memory usage, and workgroup size.
• Memory Coalescing: Adjacent threads should access adjacent memory addresses to coalesce into wide memory transactions (128-512 bytes per access).
• Bank Conflicts: Shared memory is banked (32 banks). If multiple threads access the same bank in the same cycle, accesses serialize. Padding shared memory arrays avoids bank conflicts.

GPU Compute Shaders are the interface between the programmer's parallel algorithm and the GPU's massively parallel hardware — providing the abstraction that makes thousands of ALUs accessible for general-purpose computation without requiring knowledge of the underlying hardware microarchitecture.