Thermal Fundamentals & the Density Wall

Every accelerator is, thermodynamically, a space heater that happens to do arithmetic. A 1,200 W GPU turns essentially all of its electrical input into heat. That heat must leave continuously and within a few degrees of a fixed temperature limit, or the silicon throttles, ages or fails. It is the one constraint in the building that does not negotiate. You can oversubscribe a fabric, defer a redundancy tier or re-price a power contract, but you cannot argue with the second law. The heat leaves through the path you built for it, at the rate physics allows, or the machine slows to match the path you actually have.

Part 5 builds on this foundation. It starts with heat-flux first principles and the thermal-resistance stack-up from junction to coolant, then follows the 2020–2027 rack roadmap while testing conventional air, close-coupled air, RDHx/AALC, hybrid and DLC against specific equipment and facility limits. The thermal metrics (approach temperature, NTU/effectiveness, airflow and pressure, coolant delta-T, flow and pressure drop) become the working vocabulary. Rack kW narrows the choices. Heat split and flux, airflow and inlet limits, FWS/TCS or entering-water conditions, room rejection, climate, redundancy and the refresh tail pick the winner.

The governing quantity is heat flux, power per unit area in W/cm², not power itself. Spread a 700 W H100 SXM module's full rated power over roughly 8 cm² and division gives 85–90 W/cm². That is a scaling calculation from NVIDIA's H100 rating, not a measured die heat flux. The module's power includes heat outside the die, and a Blackwell-class package past 1 kW still needs its own cooled area and spatial heat map before you call an average a hotspot. Flux is what the cooling solution fights, because heat removal is limited by how much surface you can couple to a coolant and how steep a temperature gradient you can sustain across it.

The chip vendor hands you two numbers that anchor the budget: the temperature limit and the heat load for the selected operating mode. Both sit inside the vendor's wider thermal and mechanical limits, which also fix the supported interfaces and heat distribution. Tjunction-max is the maximum allowable on-die temperature for that component and mode. It bounds the range the part is qualified for. Separately, the part's own protection logic decides when to throttle or shut down, and running beyond its absolute limits risks damage. TDP is the thermal design power you must remove, set by the silicon and the workload. Everything between junction and coolant is the resistance stack you have to make work, using the package, TIM, plate, fluid and flow the vendor has qualified. One unforgiving relation governs it: the temperature rise from coolant to junction equals the heat removed times the thermal resistance of the path, ΔT = Q × Rθ. Fix Q (the TDP) and Tjunction-max, and the resistance you can afford collapses to a fixed budget. Spend it badly and the chip is illegal at any coolant temperature you can practically supply.

The stack is a series chain, Tjunction to Tcase to the cooling medium, and in a series circuit the largest resistor dominates. Walk it from the silicon outward and name the reference temperature beside each resistance. OCP's cold-plate qualification method uses case-to-liquid-inlet resistance, so do not add the full coolant rise again to an inlet-referenced result.

The third row shows why air loses at high component heat flux. Convective resistance from a surface to a fluid is set by the fluid's heat-transfer coefficient and the wetted area. Water's volumetric heat capacity is roughly 3,500× that of air, and its convective coefficient at a cold-plate surface is one to two orders of magnitude higher than forced air over a finned heatsink. Air can be pushed harder with more CFM, taller fins and colder supply, but each lever has sharply diminishing returns and brings a fan-power, noise and pressure penalty that can make more airflow uneconomic. Liquid does more than beat air. It shrinks the case-to-coolant resistor enough that the same TDP fits inside the same junction budget at a far more relaxed coolant temperature. → Cold-plate engineering is in Chapter 5.4; in-chip microchannels that attack Rθ-JC itself are in Chapter 16.2.

No single rack-kW figure is a physics cliff. The cooling options overlap, and each rack runs into the limits of its own equipment and facility. Where earlier Parts say “cooling cliff” or “density wall”, read both as shorthand for the point where a given rack's heat split, flux and airflow limits stop working in the hall you have. SemiAnalysis (February 2025) puts typical rear-door heat exchangers at 30–40 kW per rack and above 50 kW with active rear-door fans, both at the door maker's reference conditions. Real air and liquid-assisted systems work or fail at their own duty, set by rack airflow and inlet limits, system pressure, containment and recirculation, fan state, acoustics, design-day rejection, service access and refresh profile. So read any published rack, door or cooling-unit capacity together with the air and water temperatures, flow, pressure, fan state and heat-capture fraction it was rated at.

Three physical constraints make air unattractive as heat flux and airflow demand rise. First, on a given system curve fan power rises approximately with the cube of airflow, so more CFM can impose a steep fan-energy penalty; model it at the actual pressure and fan-efficiency point. Second, air carries little heat per unit volume: about 1.2 kJ/(m³·K), against about 4,180 kJ/(m³·K) for water. Air arrives at the server's inlet temperature and must exit under its component and exhaust limits, which caps the rise across the server; air therefore has to move heat by volume. Carrying a whole 132 kW rack on air at a 15 K rise would take about 7.3 m³/s (about 15,500 cfm) through that one rack, before fan power and acoustics are counted. Third, acoustic and velocity limits cap how hard you can blow before noise, vibration and bypass airflow make the hall unworkable and the cooling ineffective at the chip. These limits rule out conventional room air for HPE's 132 kW GB200 NVL72, whose data sheet assigns 115 kW to liquid and 17 kW to air. Its DLC requirement comes from that product's heat split, not from subtracting a universal rack-kW ceiling.

If the equipment roadmap you support may need water at the rack, reserve the hard-to-retrofit structure, routes, isolation, CDU space and rejection capacity early, even though air, RDHx, hybrid and DLC ranges overlap. Inspect an inherited air hall for those provisions rather than assuming either way. Enabled Energy estimated ~$5–6M/MW in July 2025 for repositioning an existing hall to 20–40 kW racks; adding the liquid, electrical and structural interfaces for a rack-scale conversion needs a separate budget. A different estimate, published by STL Partners in May 2026, puts a liquid-cooling retrofit at about $2M/MW, against more than $11M/MW for new liquid-cooled capacity. The release does not say what that figure includes, and workload migration, downtime and lost tenancy revenue can change the case. The decision rests on your equipment roadmap and facility limits. → Retrofit paths are engineered in Chapter 5.10; air-system qualification is in Chapter 5.2.

The density wall would be an academic curiosity if accelerators had stayed put. Per-GPU thermal design power has climbed from the A100 SXM's 400 W to the H100 SXM's 700 W to GB200's ~1.0–1.2 kW. Pegatron rates each Rubin GPU in its Vera Rubin (VR200) rack at 2.3 kW, and Rubin Ultra has no published per-package TDP. Multiply by the GPUs packed into a rack and the rack-level curve is steeper still, because the scale-up domain grew at the same time and concentrated more silicon behind a single liquid manifold. Across the reference designs cited here, the move from H100-class air-capable servers to the GB200 NVL72 changed the feasible set: the OEM heat split now sends most of the rack's heat to liquid and the rest to air. Vera Rubin extends the liquid side, with fanless compute and NVLink switch trays, but the hall still needs air for configuration-specific loads such as DGX power-shelf PSUs, and NVIDIA's DSX design keeps CRAHs for other equipment (air-duty bounds in Chapter 5.2).

The rightmost column shows what each OEM profile requires. Rising density and heat flux narrow the options, but the final choice still depends on the liquid heat fraction, airflow and inlet limits, water availability, TCS/FWS conditions, climate and rejection, residual room heat, serviceability, redundancy and the future tail. The real decisions are when to cross, for which funded refresh generation, and whether the substrate is ready when you do. Reserve infrastructure for those configurations and price later generations as options. A hall scoped for a previous density can still take a new IT fit-out. The substrate is the hard part: the floor, the power chain and the cooling plant are costly and disruptive to change under an operating hall, often needing outages, displaced tenants or replaced plant. That is why Chapter 1.1 prices the ramp scenarios against the substrate before the IT is ordered.

Map the density curve and the required thermal interfaces onto the available technologies and you get an overlapping ladder. Air, rear doors, cold plates and immersion each hand the load onward when their pressure, temperature or service limits run out. Choose the lowest-cost supported rung that clears peak duty in normal and degraded operation, because every added interface costs capital and service effort you do not get back. Added liquid plumbing spends integration effort; staying on air spends fan pressure, floor area and inlet margin.

• Air (containment + CRAH/in-row). Saturates when the rack's airflow, pressure and inlet limits run out; that limit belongs to the configuration, not to a universal kW/rack cliff. It needs no facility water at the rack and has the simplest service path. Keep it as long as the air system delivers and rejects the heat at the lowest cost. It remains the right answer for air-qualified storage, networking, HGX and RTX PRO nodes, and edge equipment. An NVL72-class rack that requires DLC needs its liquid path for inference as well as training: the rung follows the rack's profile, not the workload label. → Chapter 5.2.
• Rear-door heat exchangers / air-assisted liquid. Bridges the air-cooled rack's heat into water: passive doors spend server-fan pressure, while active doors add their own fans to carry more coil pressure loss. Each door's capacity comes from its own air/water schedule, not from a universal passive or active band. A liquid coil captures heat at the rack exhaust. This is the brownfield-friendly rung when a water-fed door fits the existing server airflow, or when a cold-plate L2A unit and room plant carry the full heat without building water. It saturates where the door coil can no longer extract a high enough fraction of the heat. → Chapter 5.3.
• Direct-to-chip liquid (single-phase DLC). Single-phase DLC is the mainstream 2026 path for the rack-scale accelerators it supports, as the GB200, GB300 and Vera Rubin rows above show. Across all AI chips the shift is still under way: TrendForce (August 2026) estimates liquid cooling on about 33% of them in 2025 and projects 53% in 2026 and nearly 60% in 2027, so nearly half of this year's AI chips still run on air. Cold plates sit on the GPUs, CPUs and switches, with in-rack manifolds, dripless quick-disconnects and a CDU isolating the technology-cooling loop from facility water. With the rack, fluid and distribution matched, cold plates reach the 100 kW to 200+ kW rack range. Extending warm-water loops to a Kyber-generation roadmap needs that generation's own heat, pressure and service testing. OCP's white paper Thermal Boundaries in AI Infrastructure (v1.00, 12 October 2026) projects single-phase cold plate's physics limit at roughly 4,000 W for a reticle-class die, a few product generations out. Its authors expect rising cost and complexity, rather than that wall, to move the industry off single-phase cold plate, so reserve facility-water capacity and space for whatever method follows it. → Chapter 5.4; CDU loop in Chapter 5.6.
• Immersion (single- and two-phase). Single-phase keeps the bath below boiling and owes you a wet-server service process. Two-phase boils the dielectric and owes you vapor recovery on top of that process. PFAS exposure stays a procurement issue wherever the fluid's chemistry falls inside the applicable definition. Compare auxiliary energy, wet support reactions, fluid recovery and warranty at matched duty before paying for either bath. → Chapter 5.5.
• In-chip / direct-to-silicon microfluidics. The next rung: microchannels etched into or onto the die, a lever on the in-package Rθ-JC resistance that an external cold plate cannot reach. Roadmap, not yet default. → Chapter 16.2.

Part 5 uses a small, consistent vocabulary of thermal metrics. They are pinned down here so the engineering chapters can use them without re-deriving:

• Approach temperature — the terminal temperature difference at one end of a counterflow exchanger: the stream leaving on one side measured against the stream entering on the other at that same end. For a CDU running TCS 45 °C in / 35 °C out against FWS 30 °C in / 40 °C out, the cold-end approach is 35 − 30 = 5 K. Comparing the two outlets gives −5 K, which describes the wrong relationship. A smaller approach means a more effective and more expensive exchanger, and warmer facility water that still serves the rack. It is the knob you tune at every exchanger in the chain.
• Effectiveness (ε) and NTU — effectiveness is actual heat transfer over the thermodynamic maximum; NTU is the dimensionless measure of exchanger size (conductance × area over the minimum heat-capacity rate). The ε-NTU method sizes CDU and facility heat exchangers without solving the full temperature field. Higher NTU asymptotes toward ε = 1 with diminishing returns.
• Delta-T (ΔT) — the temperature rise a coolant carries across a load. A larger ΔT moves the same heat at lower flow (smaller pumps, smaller pipes), which is why warm-water, high-ΔT design is favored. The flow-rate rule of thumb, ~1.25–2.0 L/min per kW, is OCP's cold-plate loop guideline for a 7.5–12 K coolant rise with PG25; plain water at 1 kg/L and 4.18 kJ/(kg·K) needs about 1.2–1.9 L/min per kW over the same rise. NVIDIA's DSX reference design adds a planning criterion of at least 1.5 L/min per kW of technology-cooling flow (design guide v2.0, August 2026). That is one reference design's requirement, not a universal minimum. Use the actual fluid density and specific heat for selection, then check the component minimum flow and pressure limits. CDU selection at your rack's fluid, flow and head is in Chapter 5.6.
• Heat flux (W/cm²) — power per die area, the quantity the cold plate actually fights at the hotspot, distinct from total TDP.

The facility-efficiency metrics, PUE, WUE, ITUE and TUE, sit one level up and score the whole plant rather than a single exchanger. PUE is total facility energy over IT energy; WUE is water consumed per IT energy; ITUE and TUE extend the accounting to capture fan and pump parasitics that liquid cooling reshuffles. They are defined in Chapter 15.1 and used throughout Part 5 as the scorecard for the design choices this chapter sets up.

Trace. IT heat = 4 × 100 = 400 kW; liquid duty = 4 × 80.0 = 320 kW; air duty = 4 × 20.0 = 80.0 kW. Each water stream needs mass flow 320/(4.18 × 10.0) = 7.66 kg/s, or 459 L/min for the zone and 115 L/min per rack. Both heat-capacity rates are 32.0 kW/K. Cold-end approach = 35.0 − 30.0 = 5.0 K; hot-end approach = 45.0 − 40.0 = 5.0 K. Effectiveness = 320/[32.0 × (45.0 − 30.0)] = 0.667.

Component temperature = 35.0 °C + 1,000 W × (0.0150 + 0.0250) K/W = 75.0 °C. Margin is 85.0 − 75.0 = 10.0 K, so the specified 5.0 K reserve remains. Select this operating point for the thermal design and carry the separate 80.0 kW air duty into room cooling. Drop that branch and the equipment outside the cold plates is stranded.

Flip. At unchanged resistance and component heat, the reserve is exhausted at 85.0 − 5.0 − 40.0 = 40.0 °C inlet; warmer inlet fails. At fixed TCS temperatures, raising FWS inlet above 30.0 °C also fails the assumed 5.0 K cold-end approach requirement. The new point needs a different exchanger selection or a lower heat load. OCP cold-plate qualification §1.3.2.1 defines the inlet reference. From here, Chapter 5.6 takes the heat and flow, 5.13 the external head, and 13.5 the installed acceptance test.

Choose the heat path only after the component margin, the liquid/air split and both exchanger terminals work at the required load. A warmer operating point spends temperature reserve to gain rejection opportunity; a colder one spends plant energy to buy reserve. Carry the chosen point into the fluid and hydraulic designs before buying equipment.