Thermal Fundamentals & the Density Wall

Every accelerator is, thermodynamically, a space heater that happens to do arithmetic. A 1,200 W GPU converts essentially all of its electrical input into heat, and that heat must be removed continuously and within a few degrees of a fixed temperature limit or the silicon throttles, ages, or fails. This is the one constraint in the building that does not negotiate. You can oversubscribe a fabric, defer a redundancy tier, re-price a power contract — but you cannot argue with the second law. The heat leaves through the path you built for it, at the rate physics allows, or the machine slows down to match the path you actually have.

Part 5 builds on this thermal foundation. It starts with heat-flux first principles and the thermal-resistance stack-up from junction to coolant, then follows the 2020–2027 rack roadmap while testing conventional air, close-coupled air, RDHx/AALC, hybrid, and DLC against named equipment and facility envelopes. The thermal metrics—approach temperature, NTU/effectiveness, airflow and pressure, coolant delta-T, flow, and pressure drop—become the qualification vocabulary. Rack kW narrows the feasible set; heat split and flux, airflow/inlet limits, FWS/TCS or entering-water conditions, room rejection, climate, service/redundancy, and the refresh tail select the service.

The governing quantity is not power but heat flux — power per unit area, W/cm². A 700 W H100 die spread over roughly 8 cm² runs near 85–90 W/cm²; a Blackwell-class package past 1 kW pushes toward and beyond 100 W/cm² at the hotspot. For scale, that flux rivals a nuclear-reactor fuel rod and exceeds a domestic cooktop element by an order of magnitude. Flux is what the cooling solution actually fights, because heat removal is fundamentally limited by how much surface area you can couple to a coolant and how steep a temperature gradient you can sustain across it.

The chip vendor hands you two fixed numbers and no others. Tjunction-max — the maximum allowable on-die temperature, typically ~90–105 °C for datacenter accelerators — is a hard reliability and functional limit; cross it and the part throttles, then degrades, then fails. TDP — the thermal design power you must remove — is set by the silicon and the workload. Everything between the junction and the coolant is your design space, and it is governed by a simple, unforgiving relation: the temperature rise from coolant to junction equals the heat removed times the thermal resistance of the path, ΔT = Q × Rθ. Fix Q (the TDP) and Tjunction-max, and the resistance you can afford collapses to a fixed budget. Spend it badly and the chip is illegal at any coolant temperature you can practically supply.

The resistance stack is a series chain — Tjunction to Tcase to the cooling medium — and like any series circuit, the largest resistor dominates. Walk it from the silicon outward:

The reason air lost is visible in the third row. The convective resistance from a surface to a fluid scales with the fluid's heat-transfer coefficient and the wetted area. Water's volumetric heat capacity is roughly 3,500× that of air, and its convective coefficient at a cold-plate surface is one to two orders of magnitude higher than forced air over a finned heatsink. Air can be pushed harder — more CFM, taller fins, colder supply — but each lever has sharply diminishing returns and a parasitic-fan-power penalty that eventually exceeds the heat it removes. Liquid operates in a different regime rather than merely beating air: it shrinks the case-to-coolant resistor by enough that the same TDP fits inside the same junction budget at a far more relaxed coolant temperature. → the cold-plate engineering is in Chapter 5.4; in-chip microchannels that attack Rθ-JC itself are in Chapter 16.2.

Rack cooling is selected across overlapping equipment and facility envelopes; no universal rack-kW value is a physics cliff. The cited source instead reports 30–40 kW as a typical RDHx range and more than 50 kW with active rear-door fans; those are door-reference conditions, while actual air and liquid-assisted systems close or fail at different duties according to rack airflow and inlet limits, system pressure, containment and recirculation, fan state, acoustics, design-day rejection, service access, and refresh profile. Any published rack, door, or cooling-unit capacity must therefore carry its stated air and water temperatures, flow, pressure, fan state, containment, and heat-capture conditions.

Three physical constraints explain why air becomes unattractive as heat flux and airflow demand rise. First, on a given system curve, fan power rises approximately with the cube of airflow: increasing CFM can impose a steep fan-energy penalty, and the result must be modeled at the actual pressure and fan-efficiency point. Second, air's low heat capacity forces large temperature rises and large volumes; the supply-to-return delta-T air can carry is small, and you run out of mass flow before you run out of fans. Third, acoustic and velocity limits cap how hard you can blow before noise, vibration, and bypass airflow make the hall unworkable and the cooling ineffective at the chip. These constraints make conventional room air unsuitable for the cited 132 kW rack, whose OEM record already assigns ~115 kW to liquid and ~17 kW to air. That product-specific heat split — not subtraction from a universal rack-kW ceiling — establishes its DLC requirement.

Where the supported equipment roadmap may require water at the rack, reserve the hard-to-retrofit structure, routes, isolation, CDU space, and rejection capacity early, even though air, RDHx, hybrid, and DLC envelopes overlap. An inherited air hall may or may not lack those provisions; inspect it rather than assuming. Adding missing liquid and structural interfaces later can run ~$2M/MW (cooling-only) to ~$5–6M/MW and up and still strand power or floor capacity. This is a named equipment-roadmap and facility-envelope decision, not a workload or rack-kW lookup. → retrofit paths are engineered in Chapter 5.10; air-system qualification is in Chapter 5.2.

The density wall would be an academic curiosity if accelerators had stayed put. Per-GPU thermal design power has climbed from the A100's ~300 W to the H100's 700 W to GB200's ~1.0–1.2 kW, with the standard Rubin (VR200) package projected near ~2.3 kW — an analyst estimate that raised the earlier ~1.8 kW figure, not an NVIDIA specification — and Rubin Ultra higher still, with no per-GPU figure published. Multiply by the GPUs packed into a rack and the rack-level curve is steeper still, because the scale-up domain grew at the same time, concentrating more silicon behind a single liquid manifold. Across cited reference designs, the transition from H100-class air-capable configurations to the GB200 NVL72's declared liquid/residual heat split narrowed the feasible set from qualified air or RDHx paths to a supported DLC-plus-residual-air architecture.

The rightmost column is the consequence. As density and heat flux rise, the equipment profile narrows the feasible set, but selection still depends on liquid heat fraction, airflow/inlet limits, water availability, TCS/FWS conditions, climate/rejection, residual room heat, serviceability, redundancy, and the future tail. The only decisions left are when you cross (which generation your facility targets) and whether the irreversible substrate is ready when you do. A hall scoped for 40 kW air-cooled racks cannot absorb a 132 kW NVL72 generation, let alone a 600 kW Kyber-class rack — not the floor, not the power chain, not the cooling plant. The expensive mistake of the 2026 era is designing to today's density and being surprised by the ramp.

Map the density curve onto the available cooling technologies and you get a hierarchy — a ladder where each rung removes more heat at higher capital cost and integration complexity, and each rung saturates at a density that hands the load to the next. Choose the lowest rung that clears your peak density with margin, because every rung up costs money, water, and plumbing complexity you do not get back.

• Air (containment + CRAH/in-row). Saturates ~40–50 kW/rack. Cheapest, simplest, no facility water at the rack. Still the right answer for storage, networking, enterprise 8-GPU HGX and RTX PRO nodes, and edge. Frontier inference is not on that list: it lands on the same NVL72-class DLC racks as training, so the rung follows the named rack profile, not the workload label. → Chapter 5.2.
• Rear-door heat exchangers / air-assisted liquid. Bridges ~50–100 kW/rack. Captures heat at the rack exhaust with a liquid coil; the brownfield-friendly rung because it needs no chip-level plumbing and tolerates facilities without facility water. Saturates where the door coil can no longer extract a high enough fraction of the heat. → Chapter 5.3.
• Direct-to-chip liquid (single-phase DLC). A mainstream 2026 path; one reported forecast is ~55%, but PMR's published category is broader cold-plate liquid cooling rather than measured single-phase-DTC deployment share. Cold plates on the GPUs/CPUs/switches, in-rack manifolds, dripless quick-disconnects, a CDU isolating the technology-cooling loop from facility water. Clears 100 kW to 200+ kW and scales to the Kyber generation with warm-water loops. → Chapter 5.4; CDU loop in Chapter 5.6.
• Immersion (single- and two-phase). Best-in-class PUE, but niche: single-phase wins on serviceability and floor loading, two-phase stalled on the PFAS reckoning and insurability. → Chapter 5.5.
• In-chip / direct-to-silicon microfluidics. The next rung, attacking the in-package Rθ-JC resistor itself with microchannels etched into or onto the die — the only lever that touches the dominant resistance the cold plate cannot reach. Roadmap, not yet default. → Chapter 16.2.

Part 5 leans on a small, consistent vocabulary of thermal metrics. Pin them down here so the engineering chapters can use them without re-deriving:

• Approach temperature — the residual gap between the two streams leaving a heat exchanger. Smaller approach, more effective and more expensive exchanger, warmer achievable cold-side supply. The single knob you tune at every exchanger in the chain.
• Effectiveness (ε) and NTU — effectiveness is actual heat transfer over the thermodynamic maximum; NTU is the dimensionless measure of exchanger size (conductance × area over the minimum heat-capacity rate). The ε-NTU method is how you size CDU and facility heat exchangers without solving the full temperature field. Higher NTU asymptotes toward ε = 1 with diminishing returns.
• Delta-T (ΔT) — the temperature rise a coolant carries across a load. A larger ΔT moves the same heat at lower flow (smaller pumps, smaller pipes), which is why warm-water, high-ΔT design is favored. The flow-rate rule of thumb — ~1.25–2.0 L/min per kW — falls directly out of the ΔT you choose.
• Heat flux (W/cm²) — power per die area, the quantity the cold plate actually fights at the hotspot, distinct from total TDP.

The facility-efficiency metrics — PUE, WUE, ITUE, and TUE — sit one level up, scoring the whole plant rather than a single exchanger. PUE is total facility energy over IT energy; WUE is water consumed per IT energy; ITUE and TUE extend the accounting to capture fan and pump parasitics that liquid cooling reshuffles. These are defined canonically in Chapter 15.1 and used throughout Part 5 as the scorecard for the design choices this chapter sets up; we name them here only so the cross-references resolve.