Energy Consumption

Energy consumption in AI is the electricity used to build and run machine learning systems, spanning the training of models, the inference that serves their answers, and the data-center and manufacturing infrastructure behind both. It matters for cost, for the carbon emissions that depend on how the electricity is generated, and for the practical question of whether enough power can be delivered where AI hardware is deployed. The subject is difficult because most of the important quantities are either proprietary or vary enormously with hardware, software, model size and location, so rigorous work tends to be specific about what it measured.

Public attention to AI's footprint began with Strubell, Ganesh and McCallum's 2019 ACL paper, "Energy and Policy Considerations for Deep Learning in NLP." The authors estimated energy and carbon for training several NLP models and reported that one pipeline including neural architecture search, a very compute-intensive procedure, corresponded to roughly 626,000 pounds of carbon dioxide equivalent under their assumptions, a figure widely repeated and often misread as the cost of training a single ordinary model. Later work refined the picture: Patterson and colleagues at Google (2021) argued that the choice of model architecture, hardware, data center and location can change training emissions by orders of magnitude, and Luccioni and colleagues (2022) carefully accounted for the BLOOM model, including embodied hardware emissions. Schwartz and colleagues' 2019 "Green AI" paper argued that research should report efficiency alongside accuracy.

Training is a large, one-time burst, while inference is a continuous, scaling cost, and for widely used models inference can dominate over time. A frontier training run occupies thousands of accelerators for weeks or months and finishes; serving, in contrast, repeats for every user request for as long as the model is in service, and demand grows with adoption. Which phase matters more depends on the model: a research model used rarely is mostly a training cost, while a popular assistant is mostly an inference cost. Estimates of energy per query vary widely across published analyses because they depend on model size, prompt and output length, batching, hardware and what is counted, so single per-query numbers should be treated cautiously unless the method is stated.

The efficiency levers are numerous, and they often reinforce each other. Algorithmic choices include using a smaller model that is adequate for the task, distillation, quantization to lower precision, sparse mixture-of-experts layers that activate only part of a network per token, and flash-attention and other kernels that waste less memory traffic. Serving levers include batching requests, caching, and speculative decoding. Hardware improves energy per operation each generation, though total demand can grow faster than efficiency gains, a rebound often discussed under the name Jevons paradox. Operational levers include siting in regions with low-carbon electricity, shifting flexible training jobs to times when the grid is cleaner, and improving data-center PUE and water use. Hardware manufacturing, which carries embodied emissions, argues for using accelerators well and for long lifetimes.

At the system level, the International Energy Agency has published analyses of data-center electricity demand and of AI's role in it, and the general conclusion across such reports is that data centers account for a small but fast-growing share of global electricity, concentrated in particular countries and regions where local grids can be strained. Because the numbers depend on assumptions about hardware shipments and utilization, projections are uncertain, and readers should treat any specific forecast as a scenario rather than a measurement. Policy responses so far include reporting and efficiency requirements; the EU AI Act, Regulation 2024/1689, for example, includes provisions asking providers of general-purpose AI models to document known or estimated energy consumption.

Common misconceptions run in both directions. One is that AI's electricity use is negligible, which ignores local grid impacts and growth in inference. Another is that a single headline figure, such as the one from Strubell et al., describes typical training, when it described a specific search-heavy pipeline. Carbon and energy are also not the same thing: the same kilowatt-hour has very different emissions depending on the grid, and water use is a separate measure again.

Read AI energy consumption as a budget with many hidden line items: the chip's draw is only the first entry, and the honest figure depends on training versus serving, the facility, the grid and what was counted.