Infrastructure

Infrastructure is everything a chip depends on that is not the chip: the power chain that delivers current to it, the thermal chain that carries heat away, the water and air that keep the fab clean enough to build it, and the links that connect it to every other chip. It is the half of the industry that never appears on a die shot, and it is increasingly the half that sets the limit.

The power chain converts hundreds of thousands of volts into less than one. Transmission arrives at a campus substation at grid voltage and steps down repeatedly: to medium-voltage distribution, through UPS and switchgear to 480 V in the hall, to a 48 V DC busbar in the rack, then to 12 V and finally to the sub-volt core rail by a multiphase voltage regulator sitting as close to the package as the board allows. Every one of those conversions is a transformer or a switching regulator with its own efficiency, and the losses multiply rather than add. Six stages at 97% each hand the silicon roughly 83% of what came through the door.

Current, not voltage, is what makes the last centimetre hard. Power is the product of the two, so as core voltage fell toward 0.7 V, current rose to compensate. A kilowatt-class accelerator draws on the order of 1,300 amps at the die. At that current, a single milliohm of resistance anywhere in the path burns over a watt and drops meaningful voltage, so the power delivery network — board planes, package planes, on-die grid, and the decoupling capacitance at every level — becomes a design problem on par with the logic it feeds. This is the direct reason for backside power delivery: moving the power network to the reverse of the wafer shortens the last leg and frees the front-side metal for signals.

Heat leaves through a chain of thermal resistances, and the worst link sets the ceiling. From junction to ambient the path runs through the die, the thermal interface material, the lid, the heatsink or cold plate, the coolant loop, a coolant distribution unit, and finally a cooling tower or dry cooler. Each interface contributes a resistance in kelvin per watt, they add in series, and improving any link other than the dominant one changes nothing. For years the dominant link was the heatsink-to-air interface, which is why air cooling held for so long and then abruptly stopped: accelerators passed the point where any practical fin stack could keep a junction below its limit.

Air cooling did not fail; the parts outgrew it. A socket dissipating a few hundred watts is comfortable on forced air. Around a kilowatt, air requires heatsinks and fan power that cost more in facility energy than the cooling saves. Direct-to-chip liquid moves a cold plate onto the package and carries heat away in water or a dielectric, handling well past a kilowatt per socket; immersion goes further by removing the interface entirely. AI racks accelerated this: a conventional rack drew five to fifteen kilowatts, while dense accelerator racks now run past a hundred, which no volume of moving air will serve.

Efficiency is measured as a ratio, and the ratio has nearly run out of room. Power usage effectiveness is total facility power divided by the power that reaches the computers. A PUE of 2.0 means half the electricity ran the building rather than the workload; the best hyperscale sites now operate near 1.1. That number is close enough to 1.0 that facility efficiency is no longer where the savings are — the remaining gains have to come from the silicon and the power chain, not from the air handlers. Water has become the parallel metric, because evaporative cooling trades electricity for water, and in a drought-exposed region that trade is political as well as technical.

The fab is itself an infrastructure problem, and a harsher one. A leading-edge fab consumes power on the scale of a small city and ultrapure water in volumes measured in thousands of cubic metres a day, manufactured on site to 18.2 MΩ·cm and delivered to tools with tighter purity specs than anything in a datacenter. Cleanroom bays run hundreds of air changes per hour through HEPA and ULPA filtration. Lithography tools sit on isolated slabs because floor vibration measured in microns per second will blur an exposure. Bulk gases, specialty gases and process chemicals arrive through dedicated distribution systems with their own abatement. None of this makes a transistor; all of it is required before one can be made.

Bandwidth falls off a cliff at every boundary, and that shapes the machine. On-die, data moves at terabytes per second across millimetres. In-package, HBM stacks and chiplet links over an interposer stay in the same order. Leave the package for the board and you are into hundreds of gigabytes per second over PCIe or CXL. Leave the chassis for the rack fabric, then the cluster network, and you are down to hundreds of gigabits. Each step outward costs roughly an order of magnitude in bandwidth and adds latency, which is why the physical arrangement of an AI cluster is an architectural decision rather than a facilities one, and why co-packaged optics is being pursued: it moves the electrical-to-optical conversion inside the package so the signal crosses the expensive boundary as light rather than as copper.

The binding constraint has moved to the grid. For most of the industry's history the question was whether a design could be built. For large AI campuses the question is now whether a hundred or more megawatts can be delivered to a specific piece of land within the schedule — and grid interconnection queues in many regions are measured in years. That has pushed operators toward siting near generation, signing long-term power agreements, and in some cases building generation directly. It is a genuine inversion: silicon used to wait for fabs, and now compute waits for electricity.

Read infrastructure through a *chain* lens rather than a *component* lens: power, heat and data each traverse a series of stages, every stage takes a cut, and the system is governed by whichever stage is worst rather than by how good the others are. Buying a better cold plate changes nothing if the interface material above it dominates, and a more efficient regulator is irrelevant if the campus cannot get its next fifty megawatts. Infrastructure is the discipline of finding the binding link and moving it.