Direct-to-Chip Liquid Cooling (DLC) — Architecture, Fluid & Thermal Design

By 2026 the question of whether AI racks need liquid is settled. GB300 NVL72 shipped in 2025 and deploys through 2026 alongside GB200; Lenovo specifies a 135 kW rack TDP and up to 155 kW peak. The best-documented heat split still comes from HPE’s GB200 NVL72 data sheet (October 2026): roughly 115 kW to liquid and 17 kW to residual air (Chapter 5.2). This chapter works from that split and scales it to your rack. A split like that makes direct-to-chip liquid cooling (DLC) plus residual room-air removal the supported basis; Chapter 5.1 covers the wider selection range. Rear-door heat exchangers and air-assisted liquid (Chapter 5.3) can stretch a brownfield hall if the door or sidecar is rated for the rack’s heat under your water and air conditions, and if the room plant can take the heat a sidecar returns to hall air. That rating rarely survives the next GPU generation. For a greenfield site whose chosen or roadmap rack needs liquid at the source, put DLC and its residual-air path in the design basis before steel is cut. For VR200 NVL72, size irreversible infrastructure to NVIDIA's 330 kW cabinet TDP facility design basis, and run energy and TCO models on Pegatron's 188 kW Max Q and 228 kW Max P operating profiles. Vera Rubin NVL72 racks have been shipping since June 2026.

Once DLC is the basis, the decisions that remain are concrete. Choose a two-phase system whose refrigerant falls under the applicable PFAS definition and you inherit that supply-chain and liability problem on top of the pressure, vapor and OEM-support obligations. Choose hand-mated flexible hoses over blind-mate manifolds and you trade factory-aligned tray geometry for field access and routing freedom, with a different leak-risk profile, although both can self-seal. Set the cold-plate delta-T too tight and you over-size the CDU and pumps; too loose and you throttle the GPUs. Each fork is cheaper to see before it is poured into a slab.

A direct-to-chip cold plate is a sealed metal block, typically copper and sometimes copper on aluminum, pressed onto the die package through a thermal interface material (TIM). Coolant is forced through internal microchannels or skived fins directly over the hot silicon. The heat path is short and its thermal resistance low, which is what makes 1.0–2.3 kW per GPU package tractable. The resistance budget tightens as power rises, since a single junction-to-inlet resistance must span the package, the TIM, the plate and the coolant’s own rise. A 1 kW part with 40 K between inlet and junction limit can spend 0.04 K/W. A 2.3 kW package (Pegatron’s TDP-support rating for Vera Rubin) on 45 °C coolant, against an illustrative 85 °C junction limit, has about 0.017 K/W. Each generation therefore needs a better plate and TIM or a cooler inlet; Chapter 5.1 sets out the thermal budget. The first fork is whether the coolant changes phase inside the plate.

Single-phase cold plates keep the coolant liquid throughout. A water/glycol mix enters, warms and leaves still liquid. At the OAI system-loop level, OCP's March 2023 OAI System Liquid Cooling Guidelines recommend a 7.5–12 °C rise, with 10 °C as a typical design target for PG25 coolant (OCP OAI guideline, §1.3). Heat removal scales with mass flow times specific heat times delta-T, so carrying more watts means pumping harder or running a wider delta-T. The design is mechanically simple, the fluids are benign (water-based, non-PFAS), the pressure regime is well understood, and it maps cleanly onto the CDU-and-secondary-loop architecture of Chapter 5.6. The penalty is that water has finite heat capacity, so very high heat fluxes demand high flow, and with it pump energy and pressure drop.

Two-phase cold plates use the latent heat of vaporization: a low-boiling-point dielectric enters as liquid, boils inside the plate and leaves as a vapor-liquid mixture. Latent heat dwarfs sensible heat, so two-phase moves high heat flux at low flow and, with a pure or azeotropic fluid, holds a nearly isothermal plate surface. That makes it an attractive candidate for the 1.5–2.3 kW package problem when its component temperature, flow stability and rejection conditions beat the supported single-phase alternative. The catch is the working fluid. The engineered dielectrics that boil at convenient temperatures have so far included fluorochemicals in the PFAS family under the broad EU proposal’s definition, which is a regulatory and liability exposure rather than a footnote. The 2026 HFO alternatives are fluorinated too, so check the composition, warranty and supply of the fluid you are buying instead of trusting the phase label.

Getting coolant from the rack inlet to the GPU, CPU and switch cold plates on 18 compute trays and 9 switch trays and back, while a technician swaps a failed tray in minutes without draining the rack, is the second major fork in DLC design. Every NVL72-class rack carries a vertical pair of manifolds, a supply rail and a return rail, which are the in-rack analogue of a busbar and run the rack height. Each tray taps the rails through couplings. The decision is which kind of coupling, and it trades serviceability against reliability and leak risk.

Blind-mate / floating-tray couplings are built into the tray and the manifold, so sliding the tray home engages the fluid connection automatically: no hose to route, no fitting to hand-torque. The floating geometry absorbs the tolerance stack so the connection self-aligns. This is the factory-integration path. Couplings are validated at L10/L11 integration (Chapter 5.13 on the mechanical side; rack integration in Part 7), and field service becomes a slide-out, slide-in operation. The cost is rigidity: rack and tray geometry are co-designed and far less forgiving of field improvisation.

Hand-mated flexible-hose quick disconnects put a short hose with a dry-break coupling between the tray and the manifold. The technician connects the two halves, and the dry-break valve seals both sides on disconnect, so the spill is limited to a small qualified release rather than an open stream. Test spill, cycle life and air inclusion at the rated pressure, following your actual service procedure. This suits the messy reality of a live hall and tolerates tolerance stack-up, but every manual connection is a potential leak point and a human-error surface, and the hoses add pressure drop and clutter. OCP now publishes two coupler families, UQD/UQDB and NVIDIA's MQD/MQDB, each interchangeable across suppliers only within its own family and dash size, and neither specification covers mating one with the other.

OCP published NVIDIA's MGX 1.2 specification (revision 1.2.0) in October 2026. It targets racks up to 250 kW: blind-mate liquid connections across the rack, compute trays with no fans, and a 5,000 A busbar that is itself liquid-cooled. Its compute trays blind-mate on UQD-08/UQDB-08 from OCP's UQD v2.0.0 (effective 22 September 2026, sizes 02 to 08). The busbar coolant connection and fanless I/O bays use NVIDIA's MQD/MQDB 1.0.0 (OCP Steering Committee approval 8 October 2026), rated to 100 psi and 17–65 °C with at most 0.025 mL lost per cycle. As with any OCP interface, the purchase order should name the revision and the mating pair that was tested.

Ask the supplier for the exact connector, manifold and rack-interface revisions and the mating pair they tested. The OCP Cold Plate index separates accepted contributions from work in development; the Open Rack index identifies the rack documents. An OCP interface name does not prove that parts from different vendors interchange, that their seals are compatible or that the warranty covers the mix. Freeze those interfaces in the purchase schedule and rehearse a real tray replacement before you approve the service boundary.

One conservation equation governs the thermal budget: the heat a loop carries equals mass flow times specific heat times the coolant temperature rise (Q = ṁ · cp · ΔT). DLC design is a negotiation among those three terms, against a pressure-drop ceiling the pumps and CDU must overcome.

Flow from the heat balance. Apply Q = ṁ·cp·ΔT to the liquid-captured heat. For water-like properties, 1 kW at a 10 K rise needs about 1.44 L/min; recalculate with the approved fluid at its operating temperature. OCP’s OAI System Liquid Cooling Guidelines (§1.3, March 2023 update) recommend a 7.5–12 °C rise for PG25 cold-plate loops, about 1.25–2.0 L/min per kW, with 10 °C and 1.5 L/min per kW as the typical target. That is a design guideline for the technology cooling system, not an OEM acceptance requirement. For the HPE GB200 NVL72’s ~115 kW liquid share, water-property illustrations come to about 82 L/min at 20 K, 165 L/min at 10 K, and ~140–220 L/min across OCP’s 7.5–12 °C range; PG25 at OCP’s per-kW rule needs ~144–230 L/min. Size the manifold and CDU from the vendor’s flow schedule for your rack, or from a calculation with your approved fluid, plus pressure and control margin. A wider delta-T cuts flow and pump energy for the same watts, but it raises the return-water temperature the heat-rejection plant must handle and pushes the warmest cold plates closer to the throttle line.

Wide versus tight delta-T. A wider coolant delta-T helps the facility: less flow, smaller pipes, lower pump energy and warmer return water that free-cooling and heat-reuse plants like (Chapter 5.7, Chapter 5.9). In QCT’s QoolRack reference for GB200 NVL72, 45 °C maximum liquid inlet and 65 °C maximum liquid return are separate limits, not a prescribed 20 K operating rise; ASHRAE W45 describes FWS supply capability. The junction budget still binds. The delta-T you can run is bounded by the supply temperature plus the cold plate’s thermal resistance, so a wide delta-T at a warm inlet can leave the last plate in a series path too warm. Designers therefore favor parallel manifold paths, where every cold plate sees near-inlet coolant. Series routing cuts the total flow the manifold carries and simplifies the run, but plate pressure drops add along the string and every downstream chip starts warmer. Choose it only where the required head and the last plate’s junction temperature both still work at the flow that plate actually gets.

Pressure-drop budget. Pump head is shared among the cold-plate pressure-flow curve (including its fittings and quick disconnects), the in-rack manifold and headers, and the secondary loop back to the CDU. Rack-level pressure drop runs through the headers, hoses, QDs and cold plates on the limiting parallel path. Use the assembly pressure-flow curve and say what it includes, or counting the same loss twice will oversize the pump. Hoses, fittings and quick disconnects consume pump head with no thermal benefit. Finer heat-transfer channels in the plate lower thermal resistance at the price of more pressure drop, and pressure drop is pump energy, which shows up in PUE. Per-chip design is therefore an explicit trade of thermal resistance against pressure drop.

Parallel routing needs 4.00 L/min at 20.0 kPa plate-path loss. Both junctions reach 35.0 + 1,000 × 0.0400 = 75.0 °C, keeping 10.0 K of reserve. Series needs 2.00 L/min at 40.0 kPa. The first plate raises the water temperature by 1.00 × 60/(2.00 × 4.18) = 7.18 K, so the second junction reaches 82.2 °C and keeps only 2.82 K. Choose parallel to preserve the required reserve. Ideal plate-only hydraulic power is 1.33 W in either arrangement; pump efficiency and external losses set the electrical input.

The choice reverses at supply ≤85.0 − 5.0 − 40.0 − 7.18 = 32.8 °C, calculated without intermediate rounding. Running the supply that cold gives up heat-rejection opportunity to save header flow. Otherwise, select a plate with lower thermal resistance or cut the component’s duty. OCP’s cold-plate method keeps the coolant-inlet reference explicit. Use 5.1 for the thermal method and 5.13 for the complete hydraulic selection.

“Liquid-cooled” overstates what DLC does. On the GB200 and GB300 racks documented here it cools the high-flux components: the GPUs, the Grace CPUs, the NVLink/NVSwitch silicon and, increasingly, the high-power VRMs. That leaves a residual air load that a hall ignores at its peril. On HPE’s GB200 NVL72 rack, roughly 115 kW goes to liquid and ~17 kW stays on air, about 13% of the rack. Scale the split to your rack: GB300-class racks draw more power and put more of it on liquid, so an air tail sized from the GB200 record is sized for the previous generation. NVIDIA’s Vera Rubin racks go further, with liquid-cooled, fanless compute and switch trays, but the power-shelf PSUs stay on air (Chapter 5.2). Other vendors differ. Microsoft’s Maia 300 rack (October 2026) is 167 kW with liquid removing more than 80% of the heat, so up to about 33 kW may still need an air path; Microsoft lists accelerators, VRMs and switch ASICs on liquid, and memory, storage and ancillary parts on air. What remains is whatever your configuration leaves without a cold plate. The usual candidates are storage, power-distribution boards, lower-power regulators, the BMC, miscellaneous board parts, NICs on some OEM builds and, depending on the configuration, pluggable optics. Optics are a growing concern because pluggable transceiver power is climbing and they sit at the rack’s air-cooled edge, where airflow is now sparse.

The fork is how you capture the air tail, and three patterns dominate. In-rack air-to-liquid: a small rear-door or in-chassis air-to-liquid heat exchanger returns the residual air load to the same liquid loop. Provided the coil temperature and airflow are enough for the residual load, the rack exhausts neutral air and can operate as a ‘zero-air-to-room’ rack. The hall can skip a separate mechanical air plant only if another sink, or demonstrated heat reduction, covers the required failure-state duty. Hybrid containment: the hall keeps a reduced CRAH or in-row air system sized only for the ~10–20% air tail, with hot/cold-aisle containment. It is simpler to retrofit but reintroduces an air plant and its PUE. Facility air: the tail dumps into the room and the building air system catches it. Distribution, installed capacity and degraded operation must all work at your actual rack count, and the plant cost should be compared with rack coils. A hall that sizes liquid for 115 kW and forgets the 17 kW air tail will throttle on optics and DIMMs while the GPUs run cold, stranding the rack it just paid to liquid-cool. Containment strategy for hybrid halls is in Chapter 5.3; Chapter 8.3 qualifies the populated switch and port-bank cooling limits; 8.9 owns optical-channel qualification and 8.10 the packaging and replaceable-part boundary. Liquid-cooling a switch ASIC does not qualify its faceplate optics.

Cold plates have a physical limit: however good the plate, heat must still conduct from the junction through the package and across the TIM into the plate before the coolant sees it. That stacked thermal resistance, especially the TIM, caps the flux a plate can handle. In-silicon cooling removes the intermediary by etching the coolant channels into the silicon itself.

In-chip (direct-to-silicon) microfluidics routes coolant through channels roughly a hair’s width, cut into the die or the package backside, so liquid flows over the hotspots inside the chip rather than across an external plate. Microsoft’s 2025 prototype uses AI-designed, leaf-vein-inspired channel networks. In Microsoft’s lab-scale tests the channels pulled heat out up to 3x better than cold plates and cut the peak rise in silicon temperature by 65%. The gain varies with workload, configuration and chip. The logic is thermal resistance. Taking the TIM and the plate out of the junction-to-coolant path is one way to attack hotspots and 3D-stacked dies. Cold plates have not hit a fixed ceiling, though. How much power a package can push into a plate depends on its heat flux and thermal resistance, the inlet temperature, the flow and the qualified assembly. CoolIT’s June 2026 release describes a 15 kW single-phase cold plate validated with water-glycol at 1.2 L/min/kW. That is a vendor report, and it does not say what test vehicle was used. Treat microfluidics as a roadmap signal rather than 2026 production technology. The full cooling roadmap, including immersion’s role and the 600 kW–1 MW rack generation, is in Chapter 16.2.

Choose the plate topology, connection interface and service isolation together. Series plumbing saves flow only while the downstream component keeps its margin; parallel plumbing spends distribution capacity to preserve a common inlet. Buy against the complete supported assembly and the operating point it was qualified at.