gpu warp scheduling

**GPU Warp Scheduling** — the mechanism by which a GPU's streaming multiprocessor (SM) manages and interleaves execution of warps (groups of 32 threads) to hide memory latency. **SIMT Execution** - **SIMT (Single Instruction Multiple Threads)**: All 32 threads in a warp execute the same instruction simultaneously on different data - If threads take different branches → **warp divergence** — some threads are masked off, executed serially - Divergence can halve (or worse) performance **Latency Hiding** - GPU hides memory latency (hundreds of cycles) by switching to another warp - While warp A waits for data, warp B, C, D execute - Need enough active warps to keep the SM busy → **occupancy** **Occupancy** - $Occupancy = \frac{\text{active warps}}{\text{maximum warps per SM}}$ - Limited by: registers per thread, shared memory per block, threads per block - Higher occupancy = better latency hiding (usually) - But: Sometimes lower occupancy with more registers per thread is faster **Warp Scheduling Policies** - **Round-Robin**: Each ready warp gets a turn - **Greedy-Then-Oldest (GTO)**: Execute same warp until it stalls, then switch - **Two-Level**: Group warps into fetch groups **Understanding warp behavior** is essential for writing efficient GPU code — the difference between naive and optimized kernels can be 10-100x.