Server & System Integration

An AI rack is not a product you buy off a shelf and bolt to the floor. It is the output of a manufacturing pipeline that starts at silicon and ends at a benchmarked cluster, and its cost and risk depend mostly on where the operator enters it. Enter at the top and a vendor hands you a turnkey, validated NVL72 with a warranty and a phone number. Enter at the bottom and you are the systems integrator: you own the bill of materials, the firmware matrix, the burn-in scripts, the acceptance gate and every tray that fails at 3 a.m. The two ends differ in quoted price, in time-to-goodput and in who is liable when the cluster misses its acceptance target, so the entry decision sets your margin stack and decides who owns the useful-output acceptance gate.

This chapter starts with the guide's L1–L12 integration shorthand and the ODM, OEM and systems-integrator roles mapped onto it. Vendor numbering varies, so contracts must define deliverables and acceptance rather than rely on a level number to say who builds what. From there it takes up the build-vs-buy fork (DGX, HGX-OEM, ODM-direct or OCP self-design), the OCP Open Rack standards and the 2026 reference systems (HGX, MGX, GB200/GB300 NVL72, AMD Helios), factory versus field integration and the logistics of shipping a wet rack that weighs 1.4–3.2 t, the goodput-oriented acceptance gate, the separate CoWoS/HBM delivery milestones that can gate the system, and the commissioning handoff to operations. Chapter 7.13 treats the rack as a physical integration unit; this chapter is about integrating it.

The industry describes hardware integration in numbered "levels," and you need them to read any contract. The scale runs from raw components to a benchmarked cluster, and the level at which a vendor delivers is the line between "you bought a server" and "you bought a working AI factory." Numbering varies slightly by vendor, but the structure is stable. L1–L5 are component and PCB assembly (bare board, SMT placement, the GPU baseboard or UBB). L6 is the populated motherboard or baseboard. L10 is a fully assembled server that boots an OS. L11 is a fully cabled rack (compute trays, NVSwitch trays, busbar, manifolds, top-of-rack switches, in-rack network and power cabling), tested as a unit. L12 is a multi-rack cluster: cross-rack cabling, the customer's software loaded, and rack-scale benchmarks that prove it performs (DCD; AMAX; Hyperscalers, 2025). These are manufacturing levels. Do not confuse them with commissioning Levels 1–5 (Chapter 13.1), where Level 5 is the integrated systems test of the whole facility (Chapter 13.6). A contract that just says "L5" can mean either, and the scope dispute that follows is expensive.

Roles divide along those levels. ODMs (Foxconn/Hon Hai, Quanta/QCT, Wistron/Wiwynn, Inventec and Supermicro's ODM arm) build roughly L1–L6. For hyperscale NVL72 volume they have also become the main L10–L11 rack integrators. In March 2026 Foxconn told investors it expects roughly 40% of this year's AI server market. OEMs (Dell, HPE, Lenovo and Supermicro's branded line) handle L6–L11 and wrap the hardware in their brand, warranty, supply assurance and global service network. The systems integrator (the OEM, a specialist or the operator itself) owns L11–L12, where validated servers become a producing cluster. The L11/L12 boundary is where time-to-goodput is won or lost: whoever owns it owns the burn-in, the acceptance gate and the install-day risk.

Four archetypes cover the entry decision, ordered from most bought to most built. Each trades margin paid against integration risk insourced and control gained. No answer is right for everyone: it depends on your scale, your engineering depth and how much of the integration burden you can carry.

DGX / turnkey (NVIDIA DGX, or the GB-series "NVL72" sold as a system). You buy a fully integrated, factory-validated supercomputer with NVIDIA's software stack, reference fabric and warranty, and one throat to choke. A single accountable supplier can reduce integration work, but price, acceptance scope and delivery still depend on the contract, and the software and fabric still carry switching costs (Chapter 7.9). It is the right call for an enterprise standing up its first cluster, or anyone who values a single accountable vendor over unit economics.

HGX-from-an-OEM (Dell, HPE, Lenovo, Supermicro building on the NVIDIA HGX baseboard or the MGX modular server reference architecture). This is the middle path, and the volume channel for enterprise and many neocloud buyers (hyperscale volume goes ODM-direct). NVIDIA sells the HGX 8-GPU baseboard (or an MGX reference architecture), and the OEM does the L6–L11 integration, adding its own chassis, thermals, BMC, service and supply assurance. You get brand-name support and a global RMA organization without the full DGX premium. The cost is that you inherit the OEM's firmware and validation cadence and pay an integration margin the ODM-direct buyer skips.

ODM-direct (buying L10/L11 straight from Quanta, Wiwynn, Foxconn or Supermicro's ODM arm). You strip out the OEM brand margin and contract the integrator directly, often to your own spec. That can mean a lower purchase price and more control over BOM and firmware, but you move closer to owning the acceptance gate and the RMA logistics yourself. It is the default for hyperscalers and large neoclouds once volume justifies an in-house hardware team.

OCP self-design (you specify the rack against Open Compute standards and have ODMs build to it). This gives maximum design control, with the design and sustaining costs inside your organization, and you are the systems integrator. You own the design, the BOM, the firmware matrix, the burn-in scripts, the acceptance criteria and every serviceability decision. It is justified when contracted volume and service benefits cover the design and sustaining organization: a point of efficiency across hundreds of thousands of GPUs dwarfs the cost of an in-house infrastructure team. Meta, Microsoft, Google and Amazon live here.

The Open Compute Project is the standards base that makes ODM-direct and self-design viable. It turns proprietary rack designs into shared, multi-vendor specifications, so an operator can qualify a second source for the same rack from Quanta, Wiwynn or Foxconn instead of being captive to one builder. The relevant AI standards in 2026 are the Open Rack family. Open Rack V3 (ORV3), Meta-led and published in 2022, moved the industry to a 21-inch rack with a vertical DC busbar, blind-mate power, native 48 V distribution and provisions for direct liquid cooling, the form factor most current high-density AI racks descend from (OCP, 2025). At OCP 2025, Meta introduced Open Rack Wide (ORW), a double-wide standard designed for the power, cooling and serviceability demands of next-generation rack-scale AI. AMD's Helios is built on it (OCP / Meta; AMD, 2025).

The reference systems are where these standards meet silicon. HGX is NVIDIA's 8-GPU baseboard reference, the building block OEMs integrate into air- or liquid-cooled servers. It is the enterprise and on-prem workhorse for inference and small training, while frontier serving deploys on NVL72-class racks alongside training. MGX is NVIDIA's modular server reference architecture, which lets partners mix CPUs, GPUs and DPUs in standardized rack designs. GB200 and GB300 NVL72 each join 72 GPUs and 36 Grace CPUs in a liquid-cooled NVLink domain. Their facility records are separate. HPE GB200 specifies 132 kW nominal TDP, split into 115 kW liquid and 17 kW air, and Lenovo GB300 specifies 135 kW TDP and 155 kW peak. NVIDIA's reference architecture puts a full GB300 rack at up to 142 kW, and HPE recommends provisioning busways for 192 kW, so size to the OEM configuration you buy. Take mass, coolant inventory and the shipping and operating limits from that configuration (Chapter 7.13). AMD Helios is the open challenger: an ORW double-wide rack carrying up to 72 MI455X (MI450-series) GPUs, ~1.4 EF FP8 / 2.9 EF FP4 and 31 TB HBM4 at rack scale, described against OCP, UALoE scale-up and Ultra Ethernet scale-out interfaces. It is the open-standards answer to a single-vendor NVL72, but implementation and interoperability need separate proof (AMD; NextPlatform; DCD, 2025–2026). The Ultra Ethernet side shows why: UEC 1.1 (released 2026-10-12) is the current specification, and since 1.0 it has defined AI Base, AI Full and HPC profiles that differ in features such as tagging and rendezvous support. Write the UEC version and profile your NICs and switches implement into the purchase contract, and test them at acceptance.

The OCP catalogue lists Open Rack V3 Base 1.1 (December 2023), Meta Frame V3 1.3 (June 2024), and ORW Base and ORW Meta Design at 1.0.1 (effective 5 May 2026; 1.0.0 was released 28 April 2026) as separate documents. It also lists the ORV3 HPR V2 72 kW power shelf (1.0.0, June 2026; Chapter 4.6). A preliminary ORW Canister guide (0.1) was under review in April 2026 but no longer appears in the catalogue (checked 2026-10-19), so ask the rack vendor which canister document its trays follow. Pin the exact manifold, power-shelf, BBU, frame and canister revisions in the assembly schedule.

Use the ASHRAE–PNNL–NEMA AI Data Center Energy Performance Framework, released June 10, 2026, to agree where IT and facility performance limits meet. It is guidance for integrated design and operation: it certifies no rack and replaces no code. Record which workload, energy, water and recovery results your design basis requires, and carry them into the integrator's acceptance and handoff contract. Appendix A indexes the standards, with Chapter 0.4 as intake, and Chapter 13.1 opens the commissioning program.

Read the last column as the strategic axis. NVL72 is a vertically integrated, single-vendor unit: you get a validated NVLink domain and a proprietary scale-up fabric, and you accept the lock-in. Helios is the open bet: a double-wide ORW rack on UALoE scale-up and Ultra Ethernet scale-out, built to spread weight and ease service by going wide rather than tall, with each OCP implementation qualified separately. Going wide changes load distribution and service volume. Whether it fixes the floor-loading and field-service problems of a single-bay NVL72 depends on the actual point loads and swept volumes, which the next section covers. Chapter 8.2 treats the scale-up fabric choice behind NVLink vs UALink, and Chapter 7.1 the merchant-vs-captive silicon framing.

The table leaves out two hyperscaler systems you can now get at rack scale, because their vendor does the integration. Google's Ironwood TPU comes as liquid-cooled racks joined over ICI and optical circuit switches into pods of up to 9,216 chips. TPUs are mostly rented through Google Cloud, but since Q2 2026 Google also delivers TPU systems to select customers' own data centers. AWS's Trn3 UltraServer joins up to 144 Trainium3 chips over NeuronLink. AWS AI Factories place Trainium systems in a customer's building, but AWS operates them, so the L11/L12 gate and the RMA stay with AWS. Google's disclosures on its customer-site deliveries do not say who signs cluster acceptance or runs the RMA, so settle both in writing before the racks ship. With either vendor, your own contract covers the site interface: power, liquid, floor loading and the network handoff, taken from the vendor's site requirements. Chapter 7.4 compares the two systems.

Once you know who integrates, the next fork is where: at the factory (rack integration and factory acceptance finish before shipment, and site and cluster acceptance follow installation) or in the field, assembled at your site. This is the central velocity decision of the deployment, and it rests on a physical fact. By generation, a populated NVL72 weighs roughly 1.4–1.5 t as GB200 or GB300 (HPE lists its GB200 at 3,245 lb with PGW) and roughly 1.8 t as Vera Rubin, which NVIDIA puts at about 4,000 lb. Either rack also holds an internal coolant inventory that differs by OEM, and it carries thousands of in-rack cables. Take its operating mass and point loads from the OEM configuration you buy and its actual foot or wheel geometry, because a footprint average is not a point-load rating. Do not infer the coolant inventory from a flow-rate figure.

Factory integration (ship the rack whole) is the 2026 default for dense liquid-cooled systems because mis-cabling and leak risk are too high to absorb on the install floor. The integrator assembles trays, busbar, manifolds, ToR switches and in-rack cabling in a controlled environment, runs rack-level burn-in and ships a tested unit. NVIDIA's rack-scale partners factory-integrate the liquid loop and re-test at the rack level, so the rack can be deployed directly at the customer site. Dell says its factory-validated PowerRack with PowerEdge XE9812 servers can move customers from delivery to production in under 6.5 hours (Dell blog, 2026-06-01; Dell publishes no methodology). CoreWeave's June 2026 release separately confirms bring-up and system-level validation of its Vera Rubin NVL72 rack.

The cost of factory integration is logistics. You now move the rack's populated shipping mass, and the question is whether it ships wet (coolant already in the loop, factory-tested as shipped) or dry (drained for transit, then filled and leak-tested in the field). Shipping wet preserves the factory test state and shaves field commissioning time, but adds the coolant's weight, freeze, spill and shock-and-tilt risk, and stricter handling. Shipping dry is lighter and safer in transit but puts a fill-and-leak-test step back on the critical path, returning some of the floor labor and risk that factory integration was meant to remove. Distance and crew maturity usually decide it: hyperscalers with short, disciplined hauls lean wet, while operators with long international freight legs or thin field crews often ship dry and fill on site. Either way, certify factory loop integrity separately and pin the shipping state down in the contract.

Serviceability runs the other way. The blind-mate density that makes a factory-built rack quick to install also makes a single-component swap in the field intricate, so price tray-level repair before you choose the deepest integration. Chapter 7.13 covers the frame and rear-plane design that sets both.

Field integration (build the rack on site), populating an empty rack with trays and cabling it in the data hall, survives only for lower-density, air-cooled or 19-inch EIA configurations where weight and cabling risk are manageable. For NVL72-class systems it is an anti-pattern. You are doing precision liquid plumbing and thousands of cable terminations in an uncontrolled environment, against an install clock, with mis-cabling a leading and entirely avoidable acceptance failure. Chapter 7.15 covers the velocity and cabling discipline this demands.

The most expensive mistake in system integration is accepting a cluster on a power-on smoke test: "it boots, it pings, sign here." AI clusters fail in ways a smoke test never sees. A GPU trains fine for an hour and then throttles on a thermal excursion. An HBM stack has a marginal bit-error rate. An optic flaps under load. A single mis-cabled link quietly halves bisection bandwidth. The gate that catches these is goodput-oriented: a multi-day burn-in that drives the cluster at full power and measures whether it sustains its target goodput, not merely whether it powers on. Goodput is useful work delivered within the agreed quality and service limits; a training cluster also gets a separate MFU target.

The case for a measured gate is empirical, and no single duration fits everyone. Meta's Llama 3 405B snapshot logged 419 unplanned interruptions over 54 days on up to 16,384 H100s, about one every three hours, and its Table 5 attributes 148 events to faulty GPUs and 72 to HBM3 errors (Meta, July 2024). On the factory side, SemiAnalysis's October 2024 operator playbook recommends at least 3–4 weeks of factory high-temperature burn-in before deployment, one point in a published range that runs from hours to weeks (Chapter 13.8). Contract the test intensity, fleet exposure, measured discovery curve and statistical stopping rule so marginal components surface before production.

A defensible acceptance program layers tests at each level. L10 is node burn-in (thermal soak, memory test, per-GPU stress). L11 is rack-level validation (leak test on the liquid loop, power sequencing, in-rack link integrity, mis-cabling verification). L12 is cluster-level acceptance (collective-communication benchmarks such as all-reduce bandwidth, a representative training run held to a target MFU, and a sustained multi-day goodput soak). The gate is contractual: it defines the number the integrator must hit, how long the cluster must hold it and the remedy if it does not. Part 13 engineers each layer: node burn-in in Chapter 13.8, fabric validation in Chapter 13.7, and the reference training run and cluster benchmarks in Chapter 13.9; Chapter 13.1 sets the commissioning levels and governance.

It is tempting to plan around rack assembly, the visible, schedulable step. That is a mistake when the binding constraint sits two tiers upstream, in advanced packaging. In 2026 TSMC's CoWoS capacity is ramping toward ~120–140k wafers/month, against roughly a million wafers of reported allocation for the year, ~60% of it NVIDIA's (Chapter 7.7 owns the series), and HBM is the companion gate (Chapter 7.6). The binding dependency can also be the logic wafer, the board build or system acceptance, and TSMC, SK hynix, Samsung and Micron capacity disclosures are not a dated allocation to this rack. Record each predecessor's committed lot, quantity, release documents and the owner of any substitute part's qualification, with the memory and packaging gates tracked as separate lines.

The consequence for system integration is a sequencing rule: the latest unfinished dependency sets your delivery date. A flawless L11/L12 integration line standing idle waiting for accelerators is the default failure of a build plan scheduled against the wrong constraint. The operators who deploy fastest secure dated CoWoS/HBM allocations before dependent commitments, treat the accelerator delivery curve as the master schedule, and stage the powered shell, cooling plant and integration capacity to be waiting on silicon rather than the reverse. Accelerator allocation strategy belongs in Chapter 7.11 and long-lead equipment procurement in Chapter 2.3; Chapter 2.1 owns the integrated schedule.

The last act of integration is the handoff to operations: the moment the cluster stops being the integrator's project and becomes the operator's producing asset. A clean handoff is itself an acceptance gate. It transfers not just hardware but the documentation that makes the hardware operable: the as-built rack and link maps, the firmware and driver baseline, the burn-in and acceptance results, the asset-and-port inventory that feeds DCIM (Chapter 14.2), and the spares and RMA terms. Skip the documentation transfer and you own a cluster nobody can service without reverse-engineering it.

Spares, RMA and serviceability are where the goodput argument pays off. When one tray failure can interrupt synchronous work, mean time to repair on that tray is a direct multiplier on effective availability and therefore on goodput. Serviceability decisions made at integration time govern it: front-serviceable trays versus racks you must pull from the aisle; blind-mate power and liquid quick-disconnects that let you swap a tray without draining the loop; an on-site spares depot sized to the fleet's failure rate rather than a vendor's standard SLA; and an RMA path whose turnaround you measured. The build-vs-buy fork returns here. A turnkey buyer inherits the vendor's RMA organization and SLA, while the OCP self-designer owns the spares pool and the repair logistics outright, which makes the entry decision a multi-year operational commitment, not a one-time purchase. Chapter 12.2 develops the reliability math behind why repair time dominates goodput, and Chapter 9.4 covers checkpointing, the software complement that bounds the cost of each failure.

Useful throughput is 3,780,000 tokens / 3,600 s = 1,050 tokens/s, so the normal run clears the 1,000 tokens/s, 89.0% quality and 50 ms p99 gates. Recovery overruns its limit by 150 − 120 = 30 s, so the cluster is not accepted. To reverse that, the repaired system must recover within 120 s, pass the same performance, quality and defect gates again, and pass the inventory, authorized-update, failed-update-recovery and rollback checks on the restored manifest. A faster normal run cannot buy back a slow recovery.

Write diagnosis, repair and a full retest into the contract, and transfer the accepted manifest, logs, spares and support contacts to operations only after the retest passes.