Facility Water Loops & Warm-Water Cooling

The CDU isolates the technology-cooling loop at the rack from the building. Everything on the other side of that heat exchanger — the pipes in the pipe rack, the pumps in the plant room, the water that ultimately carries the GPUs' heat to the sky — is the facility water system (FWS), and it is governed by one decision that sits upstream of almost every mechanical choice in the building: how warm is the water you push to the CDUs? The supported operating range is a design-basis input and an operational control variable. Together with climate, load, approach temperatures, rejection method, redundancy and an actual heat offtaker, it influences compressor hours, water/energy tradeoffs and heat-reuse potential or a thermodynamic nuisance you pay to dump.

That setpoint — the loop temperature — organizes this chapter. We map the ASHRAE liquid classes (W17 through W45+) and what each commits you to; trace why warmer-water operation can expand economizer hours and raise heat-reuse grade where the named equipment, climate, rejection and offtake design support it; and then work the engineering the temperature choice forces downstream: loop topology and flow balancing, the delta-T discipline the whole plant lives or dies on, water chemistry and treatment over a 10–15 year life, and the fill/flush/commissioning sequence. Heat rejection itself — the chillers, towers, dry coolers, and economizers the loop couples to — is Chapter 5.8; the economics of selling the heat are Chapter 15.5. Here we engineer the water that connects them.

ASHRAE TC 9.9's liquid-cooling guidance bands the facility water loop the way its air classes (A1–A4) band the supply air: each class is named for the maximum allowable supply (entering) water temperature the equipment is rated to accept. The 5th-edition refresh renamed and extended the ladder to keep pace with AI density, adding the warm classes the industry is now standardizing on. The numbers are the upper supply limit in °C; every class shares a 2 °C lower bound to keep the loop above any plausible dew point at the CDU.

A W-class is an equipment facility-water supply-temperature capability; it does not by itself specify the heat-rejection plant or operating point. For any class, overlay the selected operating point on site dry-/wet-bulb bins, heat-exchanger and rejection approaches, load, redundancy and controls to calculate compressor, water and economizer hours. Keep FWS class separate from named-product TCS acceptance limits. The class is the upper capability envelope the plant is designed to deliver and the IT is qualified to accept — day-to-day setpoints usually sit below it (typically ~25–35 °C), holding the remainder as excursion and derate headroom. The class constrains the calculation; it does not determine mechanical scope or water use without the rest of the design basis.

The strategic content of the table is the diagonal: as you walk warmer, the compressor leaves the critical path and the waste heat becomes worth something — but the thermal margin between your supply water and the chip's junction limit shrinks. For the cited QCT GB200 NVL72 reference, 45 °C maximum liquid inlet and 65 °C maximum liquid return are separate limits, not a prescribed operating pair or an ASHRAE W45 facility-loop schedule. Margin follows the selected operating point, named fluid, load, flow, CDU approach and worst branch. That is the warm-water bargain: a warmer operating point trades thermal margin for possible economizer hours; chiller scope remains a climate/load/redundancy calculation. → the junction-to-coolant resistance budget that makes this tight is Chapter 5.4; the CDU approach that consumes part of the margin is Chapter 5.6.

Warm-water cooling is the rare decision that improves two normally-opposed objectives at once, which is why it went from contrarian to default in roughly three years. The mechanism is simple thermodynamics: compressor-free rejection is feasible only when the full approach-temperature and capacity calculation closes at the applicable dry- or wet-bulb condition. An 18 °C chilled loop is colder than summer air almost everywhere, so a compressor must lift the heat uphill — that is the energy, the capex, and the single largest mechanical failure mode in the building. A higher supported FWS supply point can expand direct-rejection hours. Whether a compressor or evaporative assist remains required follows the site bins, load, cooler/HX approaches, fouling margin, redundancy case and control range. The cooling plant's share of facility energy — commonly 30–40% in air halls — collapses toward the parasitic load of pumps and fans.

The second prize is the heat itself. A chilled loop returns water at ~25–30 °C — lukewarm, useful for almost nothing, requiring an enormous heat-pump lift to reach the 60–80 °C a district-heating network wants. A warmer operating point can raise return temperature, but W45 does not prescribe a 55–65 °C return; calculate it from the actual load, flow and delta-T, which is close enough to district supply that the heat-pump lift is short, the coefficient of performance is high, and the waste heat crosses from cost to commodity. This is the engineering reason the EU's heat-reuse mandates are tractable at all: Germany's Energy Efficiency Act (EnEfG) requires data centers commissioned from 1 July 2026 to hit an Energy Reuse Factor of at least 10%, rising to 15% in 2027 and 20% in 2028 — a requirement whose feasibility depends on useful-heat demand, temperature, hours, connection and the statutory calculation — not cooling label alone. → the offtake economics and the chicken-and-egg of finding a heat buyer are Chapter 15.5; the engineering of the lift is Chapter 5.9.

The facility loop's job is to move a fixed quantity of heat from a few hundred CDUs to the heat-rejection plant while holding every CDU's supply temperature within a tight band, regardless of how the IT load shifts across the hall. Two topology forks define how you do it, and each has a characteristic failure mode.

Primary/secondary (decoupled) vs variable-primary. The classic mission-critical pattern decouples a constant-flow production loop (pumps through the chillers/heat exchangers) from a variable-flow distribution loop (pumps to the CDUs) via a low-loss header or decoupler, so the rejection plant sees stable flow while the distribution side modulates to load. It is robust and easy to stage, but it carries a second set of pumps and their parasitic energy. Variable-primary-flow collapses the two into one variable loop with bypass control — fewer pumps, lower parasitic load, lower capex — at the cost of a more demanding control problem: the plant must protect minimum flow through the rejection equipment as the distribution side throttles. For a warm-water DLC plant whose whole premise is energy efficiency, the parasitic-pump saving of variable-primary is attractive; for a plant that must ride through synchronized GPU load slams without the thermal inertia of a chilled-water buffer, the predictability of decoupled flow is worth the pumps. → the transient/ride-through dimension of this choice is Chapter 5.12.

Flow balancing across the hall. A loop feeding 200 CDUs is a hydraulic network in which the nearest CDU wants to hog flow and the farthest is starved. Reverse-return piping (Tichelmann) equalizes path length so every branch sees similar pressure drop; direct-return is cheaper but needs balancing valves and commissioning effort to avoid the far racks running hot. The branch-balance target at the rack manifold is tight — on the order of ±5% — because an out-of-balance branch shows up as a hot GPU, not just an inefficiency. → the in-rack manifold side of this is Chapter 5.4; the worst-case-branch flow validation at commissioning is Chapter 13.5.

Every pipe diameter, pump size, and heat-exchanger area in the facility loop is sized against one assumed number: the design temperature rise (delta-T) between supply and return. The relationship is fixed by physics — for a given heat load, flow is inversely proportional to delta-T. Design for a wide delta-T (say 12 °C) and you move the heat with less water, so pumps and pipes are smaller and cheaper; design for a narrow delta-T and everything grows. One Dober PG25 planning illustration uses roughly 7.5–12 °C and ~1.25–2.0 L/min per kW on the secondary side. It is not a universal target; select the project ΔT and flow from the named equipment, approved fluid, heat load, pressure budget, control range and warranty.

The fork is not the design number but what happens when you fail to achieve it. Low-delta-T syndrome is the chronic disease of large chilled/warm-water plants: through three-way-valve bleed-by, oversized coils, dirty heat exchangers, or simple control sloppiness, the loop returns colder than designed, so the plant pushes more flow to move the same heat, so the pumps run harder and the rejection equipment loses efficiency — and the symptom compounds because higher flow further shrinks the delta-T. The consequence is a plant that hits its flow limit long before its thermal limit and strands cooling capacity you paid for. The discipline is to design for an honestly-achievable delta-T, use two-way valves and proper sequencing, and meter return temperature per branch so the syndrome is caught as drift rather than discovered as a capacity wall. → the metering that makes this observable is Chapter 4.12; the controls tuning that prevents the valves from hunting is Chapter 5.12.

The facility loop is a closed mixed-metal system — steel pipe, copper and brass fittings, aluminum and copper cold-plate alloys downstream of the CDU heat exchanger, stainless plates — circulating treated water or a water/glycol mix for the life of the building. Left untreated it does three things, all of which eventually end at a fouled cold plate and a throttled GPU: it corrodes, it grows biology, and (if glycol is present) it oxidizes into acid. The treatment regime is not optional housekeeping; it is the thing standing between a clean loop and a slow-motion capacity loss.

Corrosion. A mixed-metal loop sets up galvanic couples; the defense is an inhibitor package (nitrite for ferrous-dominant systems at ~600–1,200 ppm, molybdate or phosphate blends for mixed-metal circuits) holding pH in roughly the 8.5–10.5 band. Dissolved iron above ~2 ppm is the early signal of active corrosion — corrosion products are also the particulate that fouls the ~50-micron CDU filters and the cold-plate microchannels. Biology. Warm water is a thermophile incubator; a closed loop still needs biocide and biofilm control, because biofilm both insulates heat-transfer surfaces and harbors microbiologically-influenced corrosion. (The Legionella exposure lives on the open-tower side of the plant, governed by ASHRAE 188 — that is a Chapter 5.8 problem.) Glycol oxidation. Where freeze protection or biostasis mandates propylene glycol (commonly PG25), the glycol slowly oxidizes in the presence of oxygen, heat, and metal catalysts into glycolic, lactic, and formic acids; once pH falls below ~7 the inhibitors are spent and the loop turns corrosive. The 10–15 year question — how often must the TCS/FWS fluid actually be replaced — is still being answered in the field, which is exactly why monitoring matters more than any single additive.

The 2026 best practice is continuous real-time sensing of pH, conductivity, and inhibitor reserve, with monthly lab confirmation of dissolved metals — closing the data gap that quarterly grab-samples leave open, so chemistry swings are caught in minutes rather than months. The consequence of skipping it is not abstract: a corroded, fouled loop strands cooling capacity inside a building whose entire economic case is keeping GPUs at full clock.

The facility loop is a charged pressure system that must be clean at fill, leak-tight at pressure, and balanced at flow before a single GPU is energized — and the order of operations is not negotiable. Piping design follows from the delta-T and flow decisions above: pipe is sized to a velocity that moves the design flow without erosion-corrosion (typically a few m/s ceiling) or excessive pressure drop, materials are selected and dissimilar metals isolated to prevent galvanic attack, and the routing reserves pipe-rack space and knockouts for the density ramp the building will see. The structural, surge/water-hammer, NPSH, and code-basis (ASME B31.x / EN 13480 / PED) mechanics of the charged pipe are deep enough to be their own chapter — Chapter 5.13 — and the weight of charged pipe on the slab is Chapter 6.2.

Fill and flush is where AI-density loops differ most from legacy chilled water. A loop that feeds sub-millimeter cold-plate channels cannot tolerate the mill scale, flux, weld slag, and cutting debris that a new piping system always contains; left in, that debris fouls the CDU filters and the microchannels within weeks. The protocol is a staged chemical clean and a multi-pass flush — commonly a high-velocity flush to dislodge debris, a chemical cleaning/passivation pass, and repeated deionized-water flushes (factory-integrated racks see ~3x DI flushes over 12–16 hours before fill) — verified by fluid quality (turbidity, conductivity, particle count) rather than by calendar. The hydrostatic test happens before this, on the isolated pipe scope filled and vented with test fluid (Chapter 5.13); only then is the loop charged with treated fluid and vented of air (entrained air kills pump NPSH and heat transfer). At gigawatt scale the flush itself is a water event — a once-through commissioning flush can dominate year-one WUE even in a dry-cooled design — which is why closed-loop flush services that capture, treat, and reuse the water (e.g. Vertiv's PurgeRite NearZero, launched Jun 2026, vendor-claimed up to 95% less flush water) are becoming the thing to specify rather than assume. Commissioning then proves the loop end-to-end: leak-tight at test pressure, every branch flowing within tolerance under worst-case-branch conditions, delta-T achieved at load, and leak-detection response time within acceptance. → the formal L1–L5 cooling-acceptance ladder, the flush acceptance criteria, and the worst-case-branch flow test are Chapter 13.5; the leak-detection and serviceability engineering is Chapter 5.11.

The recurring facility-loop mistakes all come from treating one of its design-basis numbers as adjustable later:

• Chilled-by-default for margin you didn't price. Specifying a W17/W27 loop because the chip datasheet allows it and it feels safe — and silently committing the building to a compressor in the critical path, an evaporative tower's water bill, and waste heat too cold to sell. If the silicon accepts W45, going chilled is buying thermal insurance at the price of the plant's whole efficiency case.
• Designing a wide delta-T you can't hold. Sizing pumps and pipes for a 12 °C delta-T and then operating at 6 °C because of valve bleed-by and dirty coils — low-delta-T syndrome that doubles the flow, strands cooling capacity, and surfaces as a hot-rack wall long after the plant is built.
• Skimping the flush. Treating the fill/flush as a legacy chilled-water plumbing step rather than a sub-millimeter-channel cleanliness gate — and seeding the loop with weld slag and mill scale that fouls the CDU filters and cold plates within weeks of go-live, converting a commissioning shortcut into a chronic goodput tax.
• Treating warm water as free ride-through. Banking the chiller-less efficiency while forgetting that you deleted the chilled-water thermal inertia that used to ride through an upset — and discovering at the first CDU trip that the cooling-loss response was never validated for the selected equipment.