GPU Register Optimization addresses the critical trade-off between register availability for instruction-level parallelism and kernel occupancy, directly impacting throughput and latency hiding in GPU applications.
Register File Architecture and Limits
- Register File Size: Per-SM registers (Ampere: 256 KB, Volta: 128 KB). Shared across all resident warps. Maximum per thread varies by GPU model (255 registers Ampere, 255 registers Volta).
- Register Banking: 32 banks (one per thread in warp). Concurrent register access for all 32 threads requires bank conflict-free address patterns. Same-bank concurrent accesses serialize.
- Register Allocation: Compiler allocates registers to variables. Scalar variables (float) 1 register; double 2 registers; arrays need consecutive registers.
- Allocation Pressure: More live variables → more registers. Compiler optimizes to minimize register count (without harming ILP).
Register Spilling to Local Memory
- Spilling Mechanism: When register count exceeds budget (--maxrregcount), excess data spilled to local memory (on-chip cache hierarchy: L1 → L2 → HBM).
- Performance Impact: Local memory ~100-500 cycles latency (vs ~10 cycles for register). Spilled values effectively become cache misses. Throughput drops 10-100x.
- Spill Detection: Profiler reports spill rate (registers spilled per thread). Nonzero spill rate indicates register pressure. Target: 0 spills for performance-critical kernels.
- Reduce Spilling: Decrease --maxrregcount (fewer blocks resident, less parallelism), rewrite code to reduce register pressure (reorganize loops, fuse operations).
Occupancy-Register Tradeoff
- Occupancy Definition: Percentage of SM warp slots filled. More registers per thread → fewer resident warps → lower occupancy.
- Occupancy Curve: Register count vs occupancy (inverse relationship). Register count = N → occupancy = (SM_reg_size / (N × threads_per_warp × warps_per_block)).
- Latency Hiding: High occupancy (many resident warps) hides memory latency. Low occupancy (few register, few warps) relies on few warps to hide latency.
- Optimal Point: Often exists between extremes. Too low register (occupancy 100%) = stalls on memory. Too high register (occupancy 25%) = stalls on latency.
PTX ISA Register Model
- PTX Register Classes: %r (32-bit register), %rd (64-bit), %p (predicate), %f (float), %d (double). Abstract model (not tied to specific GPU architecture).
- Virtual Registers: PTX compilation assigns unlimited virtual registers; target-specific compiler (NVCC, PTXAS) maps to physical registers.
- Physical Constraints: Target GPU (SM 7.0, 8.0, 9.0) determines physical register count per warp, occupancy implications. Same PTX code → different occupancy on different GPUs.
- ISA Compatibility: PTX forward/backward compatible within reason. Code compiled to PTX can target multiple GPU architectures (with occupancy variation).
Compiler Register Allocation Strategies
- Register Pressure Analysis: Compiler builds interference graph (variables live simultaneously). Graph coloring assigns registers; chromatic number = min registers needed.
- Spilling Decision: When variables exceed registers, spill to local memory. Decisions impact performance; algorithm heuristic-based (not optimal).
- Loop Unrolling Effect: Unrolling increases register count (multiple loop iterations's variables live simultaneously). Trade-off: faster loop (fewer branches) vs higher register pressure.
- Optimization Passes: Multiple passes refine allocation. LICM (loop-invariant code motion), CSE (common subexpression elimination), dead code elimination reduce register pressure.
Kernel Register Count Reduction Techniques
- Refactor Loops: Break long loops into smaller loops (reduce simultaneous live variables). Example: Process array in 256-element chunks instead of full array.
- Array Privatization: Private arrays (private to thread) expensive (registers). Replace with scalars, iterate instead of bulk allocation.
- Use Functions: Inline functions increase register pressure; non-inlined functions transfer data via memory (cheaper than spilling). Trade-off: function call overhead vs register savings.
- Reduce Precision: float (1 register) vs double (2 registers). Use float where possible; promote to double only when necessary.
Warp-Level Register Sharing and Limits
- Warp Register Pool: All threads in warp share 32 registers (Ampere) conceptually. Thread i gets registers r_{iN}, r_{iN+1}, ..., r_{i*N+N-1} (N = registers per thread).
- Cross-Warp Sharing: Register file shared among multiple warps (SM occupancy). Warp 0 occupies registers 0-4095, Warp 1 occupies 4096-8191, etc.
- Bank Conflict Minimization: Register accesses within warp sequential (thread i accesses bank i). Careful allocation avoids conflicts.
Profiling and Optimization Workflow
- Nsight Metrics: "Register per Thread" metric shows allocation. "Registers per Inst Executed" indicates spilling (>4 typical, >8 severe).
- Occupancy Analysis: Nsight reports occupancy-limiting factor (registers, shared memory, threads-per-block).
- Optimization Priority: Eliminate spilling first (highest impact). Then reduce registers if occupancy < 50% (may improve performance).