energy efficiency
**Energy efficiency is useful work completed per unit energy, or equivalently the inverse energy required per useful result.** It governs operating cost, battery life, grid capacity, cooling, carbon impact, and how much AI service fits within a fixed power envelope. Relevant units include operations per joule, TOPS per watt, tokens per joule, images per joule, or joules to train or serve a qualified result; watts alone omit execution time. A professional performance claim defines workload, useful work, input and output shapes, numerical format, batch and concurrency, warmup and measurement interval, hardware and software versions, power state, correctness tolerance, and aggregation method. Peak specifications are ceilings under particular conditions; delivered behavior includes utilization, data movement, synchronization, control overhead, and tail effects. Define useful work, quality, full measurement boundary, utilization, embodied exclusions, facility overhead, and whether a rate metric or integrated energy is reported.
**Architecture, quantitative model, and operating behavior.** Efficiency improves through smaller capacitance and voltage, specialized dataflow, local SRAM reuse, lower precision, sparsity with real skipping, compression, efficient algorithms, high utilization, and minimized communication. Process scaling helps but leakage and interconnect increasingly matter. Energy equals the integral of power over time. A higher-power accelerator can use less energy if it finishes much sooner, while a low-power device can waste energy through long runtime. PUE divides total facility power by IT-equipment power; values nearer one indicate less facility overhead. Chip TOPS/W, board tokens/J, node jobs/kWh, cluster training energy, facility energy including PUE, and lifecycle carbon answer different questions. Peak TOPS/W rarely represents an entire application. Useful analysis separates arithmetic, memory hierarchy, interconnect, storage, control, and queuing. It counts operations and bytes at each boundary, identifies dependencies and reuse, estimates ideal ceilings, and then uses counters and traces to explain the gap between the model and measurement. Ratios without a clearly named numerator and denominator invite invalid comparisons. Report useful throughput together with latency distribution, utilization, arithmetic intensity, achieved bandwidth, cache hit rate, occupancy, communication time, memory capacity, power, energy per result, quality, and cost. Include median and tail behavior, sustained rather than burst operation, repeated trials, and uncertainty. A faster approximation is not equivalent unless it meets the same accuracy and service constraints.
**Implementation, hardware mapping, and bottlenecks.** Measure wall and rail power with synchronized work counters; reduce data movement; tune voltage/frequency and batch; right-size models; quantize; exploit locality; schedule for utilization; power down idle capacity; and improve cooling and power conversion. Advanced process, SRAM, HBM, chiplets, tensor engines, clock gating, DVFS, regulators, package thermals, direct liquid cooling, and efficient network optics all contribute. Moving a bit off chip can cost far more energy than a local arithmetic operation. Dividing peak TOPS by TDP, comparing different precision or quality, ignoring hosts and cooling, using average power without time, and claiming generational factors from unaligned workloads produce misleading efficiency. Begin with a correct reference and representative shapes. Profile end to end, classify the dominant resource, inspect kernel and system timelines, change one bottleneck at a time, and remeasure because optimization moves pressure elsewhere. Tiling, fusion, batching, vectorization, layout, precision, compression, overlap, prefetch, sharding, and algorithm choice are useful only when they reduce the limiting resource. The execution path spans registers, local SRAM and caches, HBM or GDDR, host DRAM, PCIe or coherent links, scale-up fabric, network, and storage. Compute units consume tensors only when compilers and kernels issue enough independent work and the hierarchy supplies operands. Package wiring, memory stacks, clocks, voltage, thermal headroom, and power delivery determine sustained limits. Frequent mistakes include quoting peak instead of achieved rates, omitting data conversion and transfer, measuring a cached toy input, timing asynchronous work without synchronization, mixing decimal and binary units, ignoring warmup or throttling, changing precision or quality, averaging away tails, and optimizing a component that is not on the critical path.
**Measurement, validation, and engineering controls.** Measure fixed-quality tasks across load, batch, clocks, temperatures, and durations; integrate energy; include idle and auxiliary allocation; report uncertainty; and test sustained rather than boost behavior. Tokens/J, images/J, FLOPS/W, total kWh, time to quality, PUE, utilization, energy by compute/memory/network/cooling, cost, and carbon intensity matter. Energy attribution by phase and component reveals whether inefficiency comes from idle gaps, memory traffic, low utilization, communication, cooling, or excessive precision. Verification combines analytical bounds, microbenchmarks, hardware counters, kernel timelines, end-to-end traces, scaling sweeps, sensitivity to batch and shape, cold and warm runs, long-duration thermal tests, correctness comparisons, fault and congestion tests, and independent reproduction. Roofline and queueing models guide diagnosis but must be calibrated against the deployed machine. Benchmark code, datasets, model and compiler artifacts, drivers, firmware, topology, clock and power settings, environment, commands, raw samples, counter traces, and analysis notebooks remain versioned. Continuous tests detect regressions in quality, latency, throughput, bandwidth, memory, power, and cost, with thresholds chosen from variance rather than a single run. Published comparisons disclose configuration, exclusions, tuning effort, measurement boundary, quality criteria, and uncertainty. Energy and carbon claims distinguish chip, IT, and facility boundaries and avoid extrapolating one benchmark to all workloads. Owners review regressions and retain evidence sufficient to reproduce decisions.
| Lever | Primary mechanism | Likely benefit | Trade-off | Required evidence |
|---|---|---|---|---|
| Lower precision | Fewer bits and denser MACs | Compute/memory energy | Accuracy/scaling | Fixed-quality joules/result |
| Locality/fusion | Avoid external bytes | Large data-movement savings | SRAM/register pressure | Byte and energy counters |
| Specialization | Remove general overhead | High workload efficiency | Flexibility/utilization | Representative workload suite |
| DVFS/power caps | Lower voltage/frequency | Better operating point | Longer runtime | Integrated energy curve |
| Utilization/batching | Amortize fixed power | More work per joule | Latency/queueing | SLO-qualified goodput |
| Cooling/PUE | Reduce facility overhead | Lower wall energy | Capital/water/operations | Facility-metered PUE |
```svg
```
**Selection and system-level application.** Choose architectures and operating points by energy to the required outcome, not peak rate; match scale to utilization and include facility effects for datacenter decisions. Mobile AI, edge sensors, datacenter inference and training, HPC, robotics, networking, storage, and sustainable computing depend on energy efficiency. Efficiency spans algorithm, model, precision, compiler, accelerator, memory, network, scheduler, utilization, power delivery, cooling, and grid. Optimization is a system exercise across algorithms, precision, kernels, compiler, runtime, accelerator, memory, interconnect, scheduler, serving policy, cooling, and facility limits. Removing one ceiling often exposes another, so architecture decisions should optimize time and energy to a useful result rather than an isolated metric. A professional performance claim defines workload, useful work, input and output shapes, numerical format, batch and concurrency, warmup and measurement interval, hardware and software versions, power state, correctness tolerance, and aggregation method. Peak specifications are ceilings under particular conditions; delivered behavior includes utilization, data movement, synchronization, control overhead, and tail effects. Report useful throughput together with latency distribution, utilization, arithmetic intensity, achieved bandwidth, cache hit rate, occupancy, communication time, memory capacity, power, energy per result, quality, and cost. Include median and tail behavior, sustained rather than burst operation, repeated trials, and uncertainty. A faster approximation is not equivalent unless it meets the same accuracy and service constraints. CFS connects this topic to semiconductor architecture, implementation, verification, manufacturing, packaging, test, and deployed AI-system tradeoffs across the platform.