energy efficient hpc computing

**Energy-Efficient High-Performance Computing** is the **systems engineering discipline that maximizes computational throughput per watt consumed — addressing the reality that modern supercomputers and AI training clusters consume 10-40 MW of electrical power (costing $10-40 million/year), where energy efficiency determines the total cost of ownership and the physical feasibility of building larger systems, driving innovations in power-aware scheduling, DVFS, heterogeneous computing, and system-level power management**. **The Power Wall** Power consumption is the primary constraint on HPC scaling: - **Frontier (ORNL)**: 1.2 EFLOPS, 21 MW — the first exascale system. - **AI Training**: GPT-4-scale training: ~25,000 GPUs × 700W = 17.5 MW for months. - **Economic**: At $0.10/kWh, a 20 MW system costs $17.5M/year in electricity alone — comparable to hardware depreciation. - **Green500**: Ranks supercomputers by GFLOPS/W. Top systems achieve 60-70 GFLOPS/W (compared to 20-30 five years ago). **Dynamic Voltage and Frequency Scaling (DVFS)** Power scales as P ∝ C × V² × f, and frequency f ∝ V. Therefore P ∝ V³ (approximately). Reducing voltage by 10% reduces power by ~27% while reducing frequency by ~10%: - **Per-Core DVFS**: Each core operates at the minimum voltage/frequency that meets its workload demand. Memory-bound phases: lower frequency (compute units idle anyway). Compute-bound phases: maximum frequency. - **GPU Frequency Scaling**: NVIDIA GPUs dynamically adjust clock frequency (boost clock mechanism) based on power and thermal limits. Workload-dependent: memory-bound kernels may run at lower clocks with equal performance. - **Power Capping**: Intel RAPL (Running Average Power Limit) and NVIDIA NVML set power caps. Hardware automatically adjusts frequency to stay within the cap. Enables predictable power budgeting. **System-Level Energy Optimization** - **Power-Aware Job Scheduling**: Schedule compute-intensive and memory-intensive jobs concurrently to balance power load across the system. Avoid scheduling all power-hungry jobs simultaneously (would exceed facility power budget). - **Node Power Management**: Idle nodes enter deep sleep (C6 state: ~2W per node vs. 300-700W active). Fast wake-up (50-100 μs) enables aggressive sleep during communications phases. - **Cooling Efficiency**: PUE (Power Usage Effectiveness) = total facility power / IT equipment power. Air-cooled: PUE 1.4-1.6 (40-60% overhead). Liquid-cooled: PUE 1.02-1.1 (2-10% overhead). Direct-to-chip liquid cooling (cold plates) is now standard for GPU-heavy AI clusters. **Algorithmic Energy Reduction** - **Communication-Avoiding Algorithms**: Reduce data movement (the most energy-intensive operation). CA-GMRES, CA-CG perform O(s) iterations between communication phases instead of O(1) — reducing communication energy by O(s)× at the cost of extra computation. - **Mixed Precision**: FP16/BF16 computation uses ~4× less energy than FP32 per FLOP. Training in mixed precision (FP16 compute, FP32 accumulate) saves 30-50% energy with negligible accuracy impact. - **Approximate Computing**: Accept imprecise results where acceptable (iterative refinement, stochastic rounding). Reduces required precision and thus energy. Energy-Efficient HPC is **the discipline that determines whether exascale and beyond is physically and economically achievable** — the systems optimization that ensures compute-per-watt improvements keep pace with compute demands, making billion-dollar computing infrastructure sustainable.