Energy Efficiency: Cooling, Free Cooling, Setpoints & Power-Chain Losses

Efficiency in AI data centers is usually discussed in the wrong units. People argue about PUE as if it were a scoreboard, when it is a consequence: the visible residue of a dozen upstream decisions about coolant temperature, economizer design, setpoint discipline, and how many conversion stages sit between the utility and the GPU's voltage regulator. Most of the leverage is spent at design time, in decisions that are expensive or impossible to reverse once steel is cut and water is plumbed.

Three things changed with the AI buildout. First, the overhead shrank. Air-hall cooling typically ran at about a quarter to two-fifths of IT load. A well-designed warm-water direct-to-chip (DLC) plant can bring total facility overhead toward 5–10%. The marginal efficiency game has moved off the chiller and onto the coolant loop and the power chain. Second, density removed the slack. Vendor designs now span Supermicro's 60 kW air-cooled HGX B300 layout, HPE's 132 kW nominal GB200 NVL72, NVIDIA's reference architecture for a full GB300 NVL72 rack at up to 142 kW, and NVIDIA's 330 kW Vera Rubin cabinet design basis. At those densities you cannot waste a few points of efficiency on conservative setpoints: the cooling plant is on the critical path, and every avoidable watt of overhead is a watt of IT capacity you cannot energize at a power-bound site. Third, the metric stopped meaning what it used to: PUE rewards moving losses inside the IT boundary, the critique in Chapter 15.1. That chapter owns the metric stack; this one covers the physical decisions that move it.

Draw the energy budget first; the leverage shows up in it. Reference breakdowns of air-cooled facilities put cooling at roughly 55–70% of non-IT energy. Transformer, UPS and distribution losses, the power chain, account for most of what remains, and lighting and building systems for a small remainder. Warm-water DLC cuts the cooling share. In a DLC hall the power chain is a comparable slice of a far smaller overhead. In an air-cooled hall, cooling is the dominant lever, and within cooling the dominant sub-lever is how many hours per year you can avoid running the compressor at all, not the chiller's efficiency at full load. The highest-value efficiency decision is therefore a coolant loop warm enough that the chiller is mostly off, not a better chiller. Where you still buy chillers, the refrigerant is no longer a free choice. Under US EPA rules, a chiller for data-center IT cooling installed on or after 1 January 2027 needs a refrigerant with GWP below 700. From 2027, new chillers sold in the EU also face the F-gas Regulation's limits on high-GWP refrigerants (Chapter 5.8).

The thermodynamics explain the leverage. Mechanical (compressorized) cooling is the expensive mode. Free cooling, which rejects heat directly to ambient air or water without a vapor-compression cycle, is nearly free by comparison. Two numbers you choose at design time set almost entirely how much of the year you spend in each mode: the coolant/facility-water supply temperature and the approach temperature of your heat-rejection equipment relative to ambient. Raise the supply temperature and thousands of compressor-hours become economizer-hours. That is why a Reykjavík facility and a Phoenix facility built to the same nameplate PUE can have wildly different annualized PUE. The nameplate is a snapshot at design conditions; the annualized number is the integral over the weather. Siting sets the weather (Chapter 3.7), and setpoint exploits it.

Cooling architecture is the first fork. It selects the heat-rejection path and the compatible rack densities before you tune a single setpoint, and climate, loading and plant curves then determine annual PUE. The choice comes in a small set of discrete regimes, not a smooth dial, each with its own parasitic load and a density fit set by the specific OEM configuration (the cooling cliff of Chapter 5.1). The table below lists the physical mechanisms and equipment limits. Compare annual energy at the same climate, load and useful service before choosing; cooling architecture alone gives no portable PUE ranking.

Rack density and vendor qualification limit which architectures you can choose (the left of the table). The parasitic-energy terms sit to the right, and free-cooling exposure is the lever the rest of this chapter pulls. Running air near its qualified limit instead of warm-water DLC keeps fan power high and narrows the economizer window, and climate and setpoints then set what that costs in annualized PUE. Cooling architecture is, in effect, a decision about how many compressor-hours you have signed up to pay for over the building's life.

Free cooling, or economization, rejects heat to the environment without running a vapor-compression cycle. It is the largest single annualized-efficiency lever in the building, and it comes in three architectures that trade water, capital and reachable hours against one another. The fork is whether the compressor-free hours you can reach repay the economizer’s installed and maintenance cost, and then which kind, because each couples to a different siting constraint and a different downstream cost.

Airside economization pulls filtered outside air directly into the hall when ambient is cool and dry enough, and exhausts the hot air instead of recirculating and re-cooling it. It is the cheapest to operate and the most water-free. But it imports outdoor humidity, particulates and gaseous contaminants into the white space, a real reliability liability, and it works only when the outside air can meet the air-cooled IT inlet spec (ASHRAE A1-A4) directly. Waterside economization keeps the air loop sealed and uses a cooling tower or dry cooler to chill the facility water without the chiller, through a plate heat exchanger when wet-bulb (or dry-bulb) is low enough. It tolerates a wider climate range and keeps contaminants out, at the cost of water (evaporative towers) or a larger dry-cooler footprint and a warmer achievable loop. Dry-cooler-direct (compressor-less) operation is the warm-water DLC endgame. If your facility loop is validated for dry-cooler operation, a dry cooler can reject to ambient air across most of the year with no evaporation and no compressor: zero process water for cooling, and PUE that approaches the pump-and-fan parasitic floor.

The trade is a water-versus-PUE-versus-capital triangle, and the AI-density era has bent it decisively toward the dry-cooler-direct corner. The reason is the warm loop. Once DLC lets you capture heat at a warm-water setpoint instead of cooling air to 18–27 °C, a dry cooler becomes viable for most of the year in most temperate climates. You can then design water out of the building for cooling through dry, non-evaporative heat rejection, the near-zero-WUE condition hyperscalers now publish (Microsoft’s December 9, 2024 design announcement estimates avoiding more than 125 million liters per data center per year). The same decision governs water stewardship in Chapter 15.4, so efficiency and water cannot be optimized separately: the coolant setpoint that maximizes free-cooling hours is also the one that lets you eliminate evaporative water. The cost is footprint and capital. Dry coolers are larger, and on a hot day they cannot reach as low a loop temperature as an evaporative tower, which makes this a siting-coupled decision, not a mechanical one.

Everything above converges on one number: the facility-water supply temperature. The industry has spent decades over-cooling. It treated the 18–27 °C recommended air-inlet range as if it prescribed chilled-loop temperature, although air inlet and facility water are different measurement boundaries, and in doing so threw away both free-cooling hours and any chance of heat reuse. Warm-water DLC inverts the logic. ASHRAE's liquid-cooling classes (W17, W27, W32, W40, W45, and W+ in the 5th-edition Thermal Guidelines) are keyed to the upper facility-water supply temperature, so that the setpoint becomes a deliberate design choice. OCP’s durable-coolant roadmap, dated 1 October 2024, targets TCS coolant at 30 °C or above at the silicon so the facility can serve successive hardware generations without rebuilding for a colder requirement. Set the future tray envelope before choosing FWS temperature: subtract the CDU approach from the required TCS supply, then size the plant for that colder supported point. Chapter 5.7 makes this a facility-loop design decision. That design target is separate from NVIDIA’s Vera Rubin reference: 41 °C FWS into the CDU and 45 °C rack coolant. Neither is a universal facility-loop setpoint.

The consequence chain from this one setpoint is the most important in the chapter. A warmer loop (a) widens the temperature difference between your coolant and ambient, which (b) lets a dry cooler or economizer reject heat across more of the year, which (c) can drive compressor-hours toward zero where weather and plant curves permit, which (d) drops annualized PUE toward the parasitic floor, and at the same time (e) lifts the return-water temperature high enough that the waste heat becomes a sellable product instead of a disposal problem (Chapter 15.5). One supported setpoint, five downstream effects to test: fan and pump demand can offset compressor savings, and a hotter return earns money only when the heat buyer accepts and pays for it. The cost is margin. A warmer loop leaves less thermal headroom, so the cold-plate design, flow rate and CDU approach temperature must be tighter and the controls more disciplined (the transient-stability problem of Chapter 5.12). The engineering of the loop that delivers it lives in Chapter 5.7.

Setpoint strategy is where design intent meets operating reality, and where most facilities leave efficiency on the table. Operations teams instinctively run cold and conservative because it feels safe. But ASHRAE's recommended IT inlet range has been 18–27 °C for years, with A1–A4 allowable ranges extending to ~32–45 °C. Every unused degree can cost compressor energy, while warmer inlet air can raise server-fan power, so spend the thermal margin only after checking total facility energy and accepted work. Some high-density air-cooled servers fall under ASHRAE Class H1 instead: 18–22 °C recommended, 15–25 °C allowable. The server maker decides which class applies, and in an H1 zone you have a few degrees of headroom, not ten (Chapter 5.2 owns the inlet ranges). Setpoints are a deliberate risk-versus-efficiency trade, and you must make it explicitly rather than default to "cold."

The fork has three settings. Conservative (low IT inlet, cold loop, wide margin) maximizes ride-through and equipment-longevity headroom at the cost of free-cooling hours. It is defensible for an air hall whose equipment limits or poor thermal monitoring require that reserve, and where the tested fault response cannot support a warmer inlet. ASHRAE-recommended (mid-band inlet, moderate loop) is the safe default for most operators. Aggressive / allowable-band (high inlet within A-class limits, warm loop) maximizes economizer hours and heat-reuse grade. It is a candidate for a well-instrumented liquid-cooled facility whose OEM limits and cooling-loss tests support it, but it narrows the thermal ride-through window. That matters a great deal when a 1 kW+ GPU's response to a cooling-loss event is product- and implementation-specific and must be established by OEM data or tests (the resilience coupling of Chapter 12.2). The downstream cost of running warm is usually not efficiency but spent transient margin, so your pump redundancy, UPS-backed cooling and controls stability had better be commissioned to match. Raising setpoints without first proving ride-through is how an efficiency initiative becomes an outage.

In air-cooled halls cooling is the larger lever, but the power chain compounds across every watt the building draws. The four-stage AC chain worked in Chapter 4.1 gives about 80–87% utility-to-VRM, with a representative ~82%; that chapter owns the full loss list and counts distribution losses separately. The SemiAnalysis SST/800 VDC model (~87%, May 26, 2026) comes from a different source and is not a guaranteed improvement on that chain. A like-for-like comparison needs the same load point, included stages and final output boundary.

Two power-chain decisions carry most of the leverage. The first is the UPS operating mode. A double-conversion UPS runs at ~94–97% efficiency because it rectifies and re-inverts continuously; high-efficiency modes reach ~98–99%, but they are two different products. Bypass-style ECO parks the inverter on standby and passes utility power through, so a transfer back to the inverter costs a few milliseconds that must be reconciled against the load's ride-through requirement. Active high-efficiency modes keep the inverter energized, hold conditioning and reactive-power compensation, and transfer to double conversion without a break. Contemporary double-conversion designs also reach ~97%, so size the recoverable gain from the two candidates' measured load-dependent efficiency curves and acceptance tests, not from the mode's name (Chapter 4.5 owns the UPS architecture decision). The second is the voltage architecture. A shift from 48 V in-rack power to 800 V-class DC distribution, whether NVIDIA's unipolar 800 VDC or OCP Mt. Diablo's differential ±400 VDC (Chapter 4.7), removes conversion stages; at the higher voltage, the same power needs less copper. It is also the structural enabler for 600 kW+ racks, so the efficiency case and the density-ramp case point the same way. Both decisions are substantially irreversible: you commission a UPS topology and a voltage class once, and redoing either mid-life is a rebuild, not a tune-up.

A plant designed to be efficient at nameplate does not necessarily operate efficiently, because the building almost never sits at nameplate. AI facilities can spend long periods below nameplate, with training jobs that ramp and checkpoint and inference fleets that swing with arrivals. Use measured load bins to size the annual penalty. Cooling plant that is efficient at full load is frequently inefficient at part load, where pumps, fans and chillers run off their best-efficiency point. The part-load gap is real money, and it is where machine-learning control has earned a place in the design basis rather than as an afterthought.

The canonical result is Google's July 2016 announcement: DeepMind machine-learning recommendations for cooling-plant setpoints cut cooling energy by ~40% and PUE overhead by ~15% against a human-tuned baseline. That was an advisory system, not the autonomous closed-loop control Google described separately in 2018 with its own results. Read it as historical proof of the mechanism against a 2016 air-cooled plant, not as the savings still on the table in an already-optimized 2026 liquid plant. The gain is invisible to nameplate PUE because it lives in the part-load, multi-variable interactions a static setpoint table cannot capture. A cooling plant has dozens of coupled actuators (pump speeds, valve positions, tower fan speeds, chiller staging) and a non-linear response surface that shifts with load and weather. An ML controller searches that surface continuously, where a human operator sets and forgets.

The decision is whether to design for this, instrumenting the plant densely enough (the DCIM telemetry of Chapter 14.2) and giving the controller safe authority, or to bolt it on later against a plant that lacks the sensors and actuators to exploit it. Retrofitting observability costs far more than designing it in, so agentic and RL-based control (Chapter 14.13) belongs in the efficiency design basis, not the wish list.

One caution. An ML controller that optimizes facility PUE without a goodput constraint optimizes the wrong thing in the wrong direction. With facility losses roughly fixed, cutting IT load raises PUE, so the objective rewards keeping servers drawing power, and idle-but-powered IT flatters the ratio while producing nothing. The objective must be useful work per watt, with the GPU thermal limit as a hard constraint, never PUE in isolation.

Trace and action. Annual IT energy = 4,000 × 8 + 3,000 × 10 + 1,760 × 10 = 79,600 MWh. The table sums to 89,920 MWh of baseline facility energy and 87,516 MWh proposed. Annual PUE is 89,920/79,600 ≈ 1.13 baseline and 87,516/79,600 ≈ 1.10 proposed, with about 2,400 MWh/year saved. Carry the proposed mode into the operating budget only after useful-work and failure-state acceptance; an acquisition decision still needs its delivered-price and cost inputs below.

Flip. The cool and mild bins save 800 + 900 = 1,700 MWh. If proposed hot overhead is h MW, annual saving = 1,700 + 1,760 × (2.0 − h). Equality is h = 2.0 + 1,700/1,760 ≈ 3.0 MW. At the stressed 3.2 MW, proposed annual energy becomes 90,332 MWh, about 400 MWh more than baseline: reject that mode. The actual quality, thermal or water limit can reject it sooner. Chapter 15.1 owns the annual ratio method; Chapter 5.8 supplies the real climate and plant curves, and Chapter 13.5 proves the proposed controls.

The last efficiency lever is the most often ignored because it appears on no nameplate: match the plant's efficiency curve to the load profile you will actually run, not the design-day peak. A cooling plant or UPS that is most efficient at 100% load is the wrong plant for a facility whose measured load trace spends most hours below peak. The fix is modularity and staging: multiple smaller chillers, pumps and CDUs that stage on and off to keep the running units near their best-efficiency point, rather than a few large units running inefficiently part-loaded. The same logic governs the power chain. A UPS bank sized so each module sits in its high-efficiency band at typical load beats one sized so every module idles at 30%.

This is a design-time decision with an operating-time payoff, and it touches two threads of this guide. On the power-bound axis, part-load-efficient plant means more of your contracted megawatts reach the IT load instead of being lost to oversized, under-loaded equipment: more compute per interconnection slot. On the goodput axis, staging logic must never compromise the thermal ride-through that protects the GPUs, so part-load efficiency and resilience are co-designed (Chapter 14.7 on operational capacity/thermal management). The anti-pattern is the facility designed and benchmarked at peak, commissioned at a single efficiency point, and then operated for years at a load where that point is irrelevant: efficient on paper, wasteful in production.

Approve the energy investment on the annual bins, then prove the setpoint. Treat the roughly 800 MWh/year saving from Chapter 15.1 and the 2,400 MWh/year computed above as illustrative energy results only. For your project, take each bin’s metered energy and delivered tariff from Chapter 3.3. Subtract added maintenance, water treatment, controls and rollout costs, then price the resulting dated cash flows against installed capex using Chapter 1.8. If the saving disappears once demand charges or warm-day IT fan power enter the boundary, reject the investment.

The mechanical lead proposes the FWS and TCS supply, ΔT, flow, fluid and CDU approach for the rack you are buying. Before testing, the commissioning lead writes down the junction-temperature, IT/facility power and accepted-output limits, then records traces for design-day operation, a pump or CDU loss, sensor failure and controller fallback. Until those traces exist, keep the current setpoint and do not adopt the warmer one. The efficient normal point is no bargain if a cooling fault consumes the remaining ride-through time before the controller can shed load; Chapter 13.5 owns the acceptance test.