Appendix B
Reference Designs & Worked Examples
Pick the archetype, accelerator generation, and scalable unit, and the megawatts, flow, fiber, ports, and dollars fall out of a handful of multipliers — this appendix supplies them and works three reference builds.
What you'll decide here
- Start from the per-archetype design-basis sheet that matches your dominant workload — it fixes density tier, cooling modality, fabric blocking, redundancy, and the GPU:CPU/storage/network ratios that every later number multiplies against.
- Treat the scalable unit (SU) as the atomic costing and deployment block: size the SU once from the budget table, then multiply — campuses and clusters in this appendix are all integer counts of SUs, not bespoke arithmetic.
- Use the 50 MW campus and 100k-GPU BOM as order-of-magnitude calibrators, not bids: counts are exact from the ratios; dollar figures are 2025–2026 list/street ranges that move quarterly and must be re-quoted.
- When a vendor proposal disagrees with these tables by more than ~20% on a count (racks, CDUs, switches, optics), find out why before you sign — the divergence is usually a hidden oversubscription, redundancy, or generation assumption.
- Re-derive, do not interpolate, when you change generation (GB200 → GB300 → Vera Rubin → Kyber): density, flow, and busbar current step discontinuously, so the multipliers in §2 are generation-stamped on purpose.
This appendix is the reusable arithmetic layer behind Part 1's archetype framework (Chapter 1.1) and Part 1's requirements matrix (Chapter 1.7). It does four things, in order: (1) a per-archetype design-basis sheet that freezes the inputs every later number inherits; (2) a scalable-unit (SU) budget giving the power, cooling, water, and network draw of one atomic deployment block per accelerator generation; (3) a 50 MW campus sized from the SU up, with the power chain, cooling plant, and water loop derived; and (4) a 100k-GPU cluster reference BOM with counts and rough 2025–2026 costs for GPUs, racks, CDUs, switches, optics, and storage.
The method throughout is multiplier-first. A reference design is a chain of ratios: GPUs per rack, racks per SU, kW per rack, L/min per kW, NICs per node, optics per NIC, GB/s per GPU of storage. Once those are pinned, every aggregate is a multiplication you can audit. Counts in the tables are exact arithmetic from the stated ratios. Dollar figures are 2025–2026 street/list ranges (sources stamped inline); they drift quarterly and are calibration aids, not quotes. Density, flow, and current figures are generation-stamped because they step discontinuously across GB200 → GB300 → Vera Rubin → Rubin Ultra Kyber; do not interpolate across a generation boundary.
1. Per-archetype design-basis sheets
The design-basis sheet is the single page that everything downstream inherits — the concrete instantiation of the workload-profile and design-basis artifacts named in Chapter 1.1. Choose the row that matches your dominant archetype, and the rest of the appendix is parameterized for you. The three reference builds in §3–§4 use the frontier-training column unless noted, because it is the most constraining; an inference-shaped build relaxes density, fabric, and redundancy and is cheaper on every axis.
| Parameter | Frontier training | Post-training / RL | Online inference (frontier / enterprise) | Batch inference | Edge inference |
|---|---|---|---|---|---|
| Dominant accelerator | GB200/GB300 NVL72 | Disaggregated: NVL72 trainer + HGX rollout | Frontier: GB300 NVL72; enterprise: HGX B300 / RTX PRO | HGX B200; prior-gen acceptable | L4/L40S, Jetson, single B200 |
| Rack density (design) | 120–142 kW | Mixed: 132 kW trainer / 40–60 kW rollout | Frontier 132–142 kW; enterprise 40–60 kW | 30–60 kW | 5–50 kW networked edge DC class |
| Cooling modality | DLC mandatory, warm-water | DLC trainer + RDHx/air rollout | Frontier: DLC mandatory; enterprise: air or RDHx | Air often sufficient | Air / sealed modular |
| Scale-up domain | 72 GPUs (→144, →576) | 72 trainer / 8 rollout | 72 (frontier NVL72); 8 (enterprise HGX node) | 8 | 1 (single node) |
| Scale-out fabric | Measured collectives + step-time target | Derive trainer and rollout tiers separately | Measured request/KV/EP traffic + tail SLO | Measured throughput traffic + completion target | Measured local traffic + WAN/SLO boundary |
| Fabric transport | InfiniBand XDR or Spectrum-X | IB trainer / RoCE rollout | Ethernet/RoCE common | Ethernet, cost-optimized | Standard IP |
| GPU:CPU ratio | 2:1 (NVL72: 72G:36C) | 2:1 trainer / 4–8:1 rollout | 2:1 frontier NVL72; 4:1–8:1 enterprise | 8:1+ | 1:1 appliance |
| GPU:storage (BW) | ~250–400 GB/s per 1,024 GPU | Trainer like training | KV-cache tier and prefill/decode disaggregation; model load tier | Streaming object tier | Local NVMe only |
| Resilience inputs (not topology) | Checkpoint/restart loss; maintenance/fault states; recovery SLO | Trainer/rollout state models; staleness and recovery limits | Serving SLO; replica/zone/region capacity; site-state continuity | Queue/backlog and completion limits; recovery contract | Latency routing, backhaul dependence, fleet correlation and recovery |
| EDPp sizing factor | ~1.4–1.5× TDP | ~1.4× trainer | ~1.3× TDP | ~1.2× TDP | ~1.2× TDP |
| Siting driver | Cheap firm MW + cold climate | Follows dominant sub-workload | Sub-50 ms to users | Cheapest / curtailable MW | Latency budget (30/50/100 ms) |
2. The scalable unit (SU): power / cooling / water / network budget
The scalable unit is the atomic deployment and costing block — order it, integrate it at the factory (L11/L12), ship it, energize it, repeat. Sizing the SU once and then multiplying is what makes campus and cluster arithmetic tractable. We anchor the SU to the NVIDIA DGX SuperPOD GB200 reference: 8 × NVL72 racks = 576 GPUs per SU, with the full SuperPOD at 16 SUs (128 racks, 9,216 GPUs). The budget below gives one SU's draw across four generations; later sections count SUs, not racks.
The cross-generation columns exist because the multipliers step. A GB200 SU is ~1.06 MW of IT; the same 8-rack SU at Kyber density (~600 kW/rack) is ~4.8 MW — a 4.5× jump in the same floor footprint. This is the density-ramp trap from Chapter 1.1 as a budget line: the floor, water, and busbar you reserve today must survive it.
| Metric | GB200 NVL72 (2025) | GB300 NVL72 (2025–26) | Vera Rubin NVL72 (H2 2026) | Kyber NVL144 (H2 2027) |
|---|---|---|---|---|
| GPUs per SU | 576 | 576 | 576 (72/rack; 1,152 dies) | 1,152 (144/rack; 4,608 dies) |
| Rack density (TDP) | 132 kW | 142 kW | ~200 kW | ~600 kW |
| IT power per SU (TDP) | ~1.06 MW | ~1.14 MW | ~1.60 MW | ~4.80 MW |
| IT power per SU (EDPp ~1.4×) | ~1.48 MW | ~1.59 MW | ~2.24 MW | ~6.7 MW (smoothed ~30%) |
| DLC heat to liquid (~87%) | ~0.92 MW | ~0.99 MW | ~1.60 MW (100% liquid) | ~4.8 MW (100% liquid) |
| Residual air heat | ~0.14 MW (~17 kW/rack) | ~0.15 MW | ~0 (100% liquid) | ~0 (100% liquid) |
| Secondary-loop flow (water-like fluid, 10 K: ~1.43 L/min per kW of liquid heat) | ~1,316 L/min | ~1,416 L/min | ~2,288 L/min | ~6,864 L/min |
| Coolant inlet / ΔT target | up to ~45 °C (W45; ~25 °C typical) / <10 °C | up to ~45 °C (W45) / ~10 °C | ~45 °C warm-water | ~45 °C warm-water |
| Back-end NIC ports (1× 400G/GPU rail) | 576 ports | 576 ports | 1,152 (CX-9 800G) | 4,608 (CX-9/CPO) |
| Leaf switch ports consumed (back-end) | 576 (1 port/GPU rail) | 576 | 1,152 | 4,608 |
| Back-end optics (transceivers, 1:1) | ~1,152 (NIC+leaf ends) | ~1,152 | ~2,304 | ~9,216 (CPO shifts mix) |
| Storage BW attributable (~250 GB/s/1,024 GPU) | ~140 GB/s | ~140 GB/s | ~280 GB/s | ~1,125 GB/s |
Worked example: deriving one GB200 SU line-by-line
Take the GB200 column and walk it forward so the multipliers are explicit. GPUs: 8 racks × 72 GPUs = 576. IT power (TDP): 8 × 132 kW = 1.056 MW ≈ 1.06 MW. EDPp: 1.06 MW × 1.4 ≈ 1.48 MW provisioned on the rack power chain. Heat split: at ~115 kW liquid + ~17 kW air per rack, liquid carries 8 × 115 = 920 kW and air carries 8 × 17 = 136 kW. Flow illustration: keep the 920 kW liquid heat boundary fixed. With water-like properties and a declared 10 K design rise, 1.43 L/min·kW × 920 kW ≈ 1,316 L/min for the SU (about 165 L/min per rack). Recalculate with the approved fluid, operating temperature and selected ΔT; do not use the 1,056 kW total rack TDP as TCS heat. NICs: 8 racks × 18 compute nodes... note NVL72 presents 72 GPUs across 18 trays; a rail-optimized back-end gives 1× 400G per GPU → 576 ports/SU (≈230 Tb/s). Optics: the back-end optic count tracks GPU rails, not NIC bodies — each of the 576 GPUs drives one 1:1 rail link, and each link burns two transceivers (server/NIC end + leaf end), so 576 GPUs × 2 ≈ ~1,152 rail-side optics for the SU's share of the non-blocking fabric (before spine). These are the only numbers; everything in §3–§4 is integer multiples of them. → SU definition in Chapter 1.7; fabric sizing in Chapter 8.5.
3. Worked example: a 50 MW campus sized from the SU up
Now multiply. The brief: a 50 MW-class IT frontier-training reference campus on GB200/GB300 NVL72, built as integer SUs, with the power chain, cooling plant, and water loop derived. We size on TDP for the IT budget and EDPp for the electrical chain, and we reserve floor/water/busbar headroom for a GB300 → Vera Rubin density step (the irreversible substrate from Chapter 1.1).
SU count. Choose 48 SUs as a clean reference arrangement (a clean 3 × 16-SU SuperPOD-scale halls, or 6 × 8-SU halls). That is 48 × 8 = 384 NVL72 racks and 48 × 576 = 27,648 GPUs. Total IT at 132 kW/rack is exactly 50.688 MW, which exceeds a hard 50.000 MW cap by 0.688 MW. If 50.000 MW is a hard cap, use at most 47 SUs: 376 racks, 27,072 GPUs, and 49.632 MW IT.
| Subsystem | Sizing basis | Quantity / value |
|---|---|---|
| Scalable units | Reference arrangement | 48 SUs |
| NVL72 racks | 48 × 8 | 384 racks |
| GPUs | 384 × 72 | 27,648 GPUs |
| IT power (TDP) | 384 × 132 kW | 50.688 MW (48-SU reference) |
| Facility power (PUE ≈ 1.2) | 50.688 MW × 1.2 | ~60.8 MW |
| Utility interconnect (N, +margin) | ~61 MW × 1.15 | ~70 MW POI / 2× 132 kV feeders |
| Main transformers | ≥2 × 75 MVA (N+1 at MV) | 2–3 × 75 MVA |
| MV distribution | 33/13.8 kV ring or radial | per-hall 13.8 kV → 415 V / 800 VDC |
| Ride-through stack (protected-IT boundary) | 50.688 MW continuous nominal IT; any declared rack EDPp/transient is allocated across rack capacitance/BBU and upstream UPS/BESS | Do not treat a sub-second rack peak multiplier as central-UPS continuous MW |
| Backup generation (generator-terminal critical-load boundary) | 50.688 MW IT × 1.2 PUE = 60.8256 MW steady, plus explicit derating/operating reserve and auxiliaries outside that boundary | ~65–75 MW is a screening installed-capacity band only if the load schedule proves it; ride-through bridges rack-side sub-second transients |
| DLC heat to facility water | 384 × 115 kW | 44.160 MW thermal |
| CDUs (HPE 1.3 MW; maximum eight racks/CDU; local 3+1 per 24-rack pod) | 16 pods × (3 duty + 1 valved standby); each duty CDU = 8 × 115 kW = 0.920 MW | 64 installed (48 duty + 16 standby) |
| Secondary-loop flow | 384 × ~190 LPM | ~73,000 L/min aggregate |
| Heat rejection | ~61 MW total heat | towers/dry-coolers + adiabatic, economized |
| Water make-up (WUE ~0.5 L/kWh of IT energy) | 50.688 MW IT × 0.5 × 8,760 h | ~222 ML/yr (≈ 610 m³/day average; size the permit/storage on peak-day draw) |
| Back-end fabric (1:1) | 27,648 GPUs, 8-rail fat-tree | ~800–950 leaf+spine switches (scaled from §4's 101k-GPU basis) |
| Floor area (white space) | 384 racks @ ~30 m²/rack incl. aisles/CDU | ~11,500 m² + plant |
| Floor loading basis | declared equivalent-uniform screen plus OEM foot/wheel, rolling and rigging reactions | verify project load combinations against the complete slab/access-floor assembly and move route |
4. Reference BOM: a 100k-GPU GB200/GB300 cluster
The flagship build: a 100,000-GPU GB200/GB300-class training cluster, costed as a bill of materials. Built from the SU: 100,000 ÷ 576 ≈ 174 SUs; round to 176 SUs = 1,408 NVL72 racks = 101,376 GPUs (≈ 100k). At 132 kW/rack that is ~186 MW IT and ~223 MW facility at PUE 1.2. That capacity may be delivered on one qualified campus or split across sites; DCI does not imply that this illustrative logical cluster runs as one cross-site synchronous job (Chapter 8.8).
The cost column carries the heaviest caveat: GPU/system pricing is 2025 street/list (SemiAnalysis), networking and storage are practitioner ranges, and all of it moves quarterly. Use the counts as gospel (they are arithmetic) and the dollars as an order-of-magnitude frame. The GPU/rack line is the single largest block (~45–55% of cluster capex), so errors elsewhere move the total far less than the GPU line itself.
| BOM line | Count | Basis | Unit cost (2025–26) | Line cost (rough) |
|---|---|---|---|---|
| GB200/GB300 GPUs | 101,376 | 176 SU × 576 | ~$60–70k effective (incl. the cluster's network/storage/integration share) | — |
| NVL72 racks (integrated, L11) | 1,408 | 176 SU × 8 | ~$3.0–3.5M / rack | ~$4.2–4.9B |
| — (rack line includes GPUs, Grace, NVSwitch, DLC) | — | GB200 ~$45–55k/GPU all-in server | — | ~$4.6–5.6B server total |
| CDUs (HPE 1.3 MW; maximum eight racks/CDU; local 3+1 pods) | 235 installed | 1,408 racks: 176 duty; 58 full 24-rack 3+1 pods plus one final 16-rack 2+1 pod | ~$120–180k | ~$28.2–42.3M |
| Back-end leaf switches (Quantum-X800/Spectrum-X) | ~2,000 | 8-rail, ~72 GPU/leaf group | ~$120k (64×800G) | ~$240M |
| Back-end spine switches | ~1,000 | 2-tier fat-tree, 1:1 | ~$120k | ~$120M |
| Front-end / storage / mgmt switches | ~600 | in-band + OOB + storage net | ~$30–60k | ~$25M |
| Back-end optics / transceivers (800G) | ~405,000 | 101,376 GPU × ~4 (NIC+leaf+spine ends) | ~$1,000–1,500 | ~$450–600M |
| DAC/AEC copper (intra-rack scale-up) | in-rack | 5,184 NVLink cables/rack (copper, in rack price) | incl. in rack | — |
| High-perf storage (parallel FS) | ~200 PB usable | ~2 PB / ~250 GB/s per 1,024 GPU → ~200 PB / ~25 TB/s | ~$0.20–0.40/GB flash tier | ~$40–80M |
| Capacity / object tier | ~150–250 PB | data lake + checkpoints | ~$0.02–0.05/GB | ~$5–12M |
| Facility power chain (per MW) | ~223 MW facility | transformers, UPS/BESS, switchgear, gen | ~$10–15M / MW (AI-grade) | ~$2.2–3.3B |
| Cooling plant + water loop (per MW) | ~186 MW IT | CDUs counted above + rejection + piping | ~$3–5M / MW | ~$0.6–0.9B |
| Cluster total (compute + network + storage + facility) | — | GPU/rack line ~45–55% of total | — | ~$8–10B |
Sensitivity: how the three builds move when you change one input
Generation step (GB200 → Vera Rubin). A future 8-rack SU must be recalculated from that named product's liquid-captured heat, approved-fluid properties and selected ΔT; the current GB200 illustration does not supply a portable flow endpoint for Vera Rubin. The 50 MW campus at Vera Rubin density needs only ~31 SUs for the same 50 MW — but each hall now dissipates ~1.6× the heat per rack, so the cooling plant, not the floor, becomes binding. Oversubscription scenario (1:1 → 2:1). A named model cuts back-end switches and optics by ~31%—about $0.3B on this illustrative ~$10B BOM. Treat that as a sensitivity, not an inference default: accept the saving only where measured traffic, placement, failure headroom, and the step-time or tail-latency SLO validate the 2:1 tier. Dry, non-evaporative heat rejection. Drives the 50 MW campus's ~222 ML/yr water make-up toward zero, at the cost of ~+0.05 PUE (~+2.5 MW facility power) and a larger heat-rejection footprint — the WUE↔PUE trade from Chapter 15.4. Effective GPU life (3 yr → 5 yr). Does not move any count or capex line, but nearly doubles the denominator in $/GPU-hr — the dominant TCO lever, quantified in Chapter 1.8 and Appendix C.
Cite this chapter
Fehn, J. (2026). Reference Designs & Worked Examples (Chapter B). The Definitive Guide to AI Data Centers. https://aidatacenterguide.com/appendix-appendices-and-reference-data/b-reference-designs-and-worked-examples (accessed 2026-08-28).
@misc{aidc-B,
author = {Fehn, Jacob},
title = {Reference Designs & Worked Examples (Chapter B)},
howpublished = {The Definitive Guide to AI Data Centers},
year = {2026},
url = {https://aidatacenterguide.com/appendix-appendices-and-reference-data/b-reference-designs-and-worked-examples},
note = {Accessed 2026-08-28}
}