Subsystem Roadmaps 2026 → 2030 (Consolidated)

Each forward-looking section in Parts 4 through 9 ends on the same warning: this subsystem is about to replatform. Power is moving off AC. Cooling is moving off air, and single-phase liquid is already being framed as the floor rather than the ceiling. Fabric bandwidth roughly doubles each accelerator generation while memory capacity steps unevenly. The network is fighting a three-way standards war while optics migrate onto the package. Storage is being pulled into the GPU's memory hierarchy. All five sit downstream of the same accelerator roadmap, so they move in sync: five faces of the power-bound density ramp. This chapter pulls those forward pointers into one 2026 → 2030 view and sorts the moves you can defer from the ones you must commit before steel is cut.

A roadmap is a list of option premiums. Betting on the wrong GPU generation is reversible, but only at the cost of new trays, qualification, software work and downtime, and a next-generation tray is not an in-place upgrade entitlement. The expensive mistake of this era is a substrate that cannot absorb the generation after next. The floor, the water, the risers and the voltage class cannot be retrofitted cheaply once the hall is live. The chapter walks the five subsystems in the order the cascade flows (power, cooling, compute and memory, network and optics, storage) and closes on the rack and facility integration that ties them together. On the 2027–2028 roadmap the unit of design stops being the 19-inch rack and becomes the ~600 kW power-and-cooling chassis.

Buy a reservation with a release test. For NVIDIA, AMD, Google and AWS, compare the same fields: dated orderable system, power modes and facility basis, floor and service loads, coolant temperature, flow and chemistry, rack and connector interface, fabric boundary, software acceptance, and support and delivery entitlement. A public NVIDIA roadmap, an AMD complete-system quote, Google TPU regional access and AWS service availability are different offers, and none alone proves that your existing hall can accept new trays. Reserve space and routes where a bounded option premium is affordable. Release busway, CDU and IT orders only when a specific system passes the interface and service tests. Reject the option when the reserved path still needs an unapproved voltage class, coolant or structural load, because empty space cannot repair an incompatible interface. → engineering acceptance in Chapter 6.2, memory sizing in Chapter 7.6 and migration acceptance in Chapter 7.9.

Power is the subsystem under the most roadmap pressure, because two exponentials meet in it. Per-rack power keeps climbing, and grid-side scarcity is what made power the binding constraint in the first place (Chapter 16.1). The climb starts at HPE's 132 kW nominal for the GB200 NVL72. The GB300 NVL72 comes next, with Lenovo rating it at 135 kW TDP and up to 155 kW peak, NVIDIA's reference architecture putting a full rack at up to 142 kW, and HPE listing 132 kW nominal. For the Vera Rubin NVL72, Pegatron rates operating profiles at 188 kW Max Q and 228 kW Max P, while NVIDIA's DSX reference design sizes the cabinet at 330 kW TDP. The roadmap then points toward ~600 kW for Rubin Ultra/Kyber. At ~40 kW you can rectify AC inside the rack and not think about it. At MW-class scale, in-rack 54 VDC distribution hits physical and economic limits as copper and power-shelf volume crowd out compute. The roadmap answer is 800 VDC, delivered either from native facility distribution or from a row-level power rack fed by existing AC infrastructure. The same pressure reaches inside the package. Core currents in the thousands of amps push voltage regulators from beside the processor to directly beneath it (vertical power delivery), and foundries are moving power rails to the back of the wafer (backside power delivery). Chapter 7.12 covers both.

The choice is between keeping 415/480 VAC to three-phase AC-input Rubin power shelves, using a row-level AC-to-800 VDC bridge that preserves existing building infrastructure, and committing a new MW-class hall to native 800 VDC distribution. AC-fed hardware already reaches high density: Vera Rubin NVL72 uses AC-fed in-rack power shelves rated 110 kW, and NVIDIA's MGX 1.2 design, published as OCP specification 1.2.0 in October 2026, supports racks up to 250 kW on a liquid-cooled 5,000 A busbar. At 800 VDC the same power flows at lower current, so it needs less copper (Chapter 4.7 works the comparison). The SemiAnalysis May 26, 2026 SST/800 VDC architecture model puts end-to-end utility-to-VRM efficiency at ~87%. Chapter 4.1 models a four-stage AC chain separately at 80.4–86.7% (82.2% representative, distribution losses excluded). The two sit roughly five points apart, but that gap becomes recoverable capacity only when both chains are compared at the same stage boundaries and load. At 1 GW input, five verified efficiency points would deliver 50 MW more output. The catch is that the DC ecosystem is immature. The solid-state transformer (SST) is offered for medium-voltage-to-800 VDC conversion, but the benchmark usually cited is a 400 kW US DOE industrial prototype that ETH Zurich's Kolar et al. showed at INTELEC 2025, not a product rating. Credit no efficiency to it without its primary test report. For the assembly you are offered, get its listing and the local installation approval; the forecast of partial 800 VDC provisions in the 2029 electrical code gives you neither. So the decision in 2026 is whether to preserve AC infrastructure with an AC-input Rubin shelf or a row-level 800 VDC bridge, or to pay now for native 800 VDC distribution where the MW-class roadmap justifies it.

Underneath distribution sits the generation question. The interconnection wall has pushed behind-the-meter (BTM) on-site generation from a fringe tactic to a mainline strategy. Cleanview tracks ~90 GW of announced BTM generation at 59 US data centers by mid-2026, largely gas turbines and engines, but only ~1 GW is under construction and ~2 GW is operating. An on-site plant may beat the utility's service date when its turbine, fuel, permit and commissioning schedule finishes first. Neither Enverus's ~40-month estimate for a new aeroderivative turbine (September 2026) nor a four-to-seven-year utility queue is a portable project promise. So power is no longer just a distribution problem inside the fence. It is a generation, fuel-supply and grid-interaction problem, and the building must be able to host the generation it may need to bridge to grid power.

Grid behavior now has a common specification. OCP's Interface Specification Between Hyperscalers and Utility Grid for Disturbance Performance v1.0.0, written by Google, Meta, Microsoft and NVIDIA, took effect on 2026-09-17. It requires computing load to stay connected through a voltage sag for a minimum time, set by the depth of the sag, before it may transfer to backup. Once the voltage recovers, 90% of the pre-fault computing load has to return within 2 s. It is an industry specification rather than a regulation. The rules in force are regional: ERCOT's NOGRR282 and SPP's large-load requirements. NERC's computational-load standards passed final ballot in October 2026, set no ride-through curve, and still await NERC Board adoption and FERC approval (Chapter 4.10, Chapter 13.6). → DC architecture in Chapter 4.7; on-site generation in Chapter 4.8; the queue and speed-to-power in Chapter 3.2.

Cooling reached its decisive fork earlier than the other subsystems, so the roadmap has a settled near term and a contested far term. Single-phase direct-to-chip liquid cooling is a mainstream 2026 path. One market forecast (Persistence Market Research, 2026) puts cold-plate cooling at more than 55% of the data-center liquid-cooling market. That category is all cold-plate cooling, not a measured share of single-phase direct-to-chip deployments. Dense NVL72-class racks need direct-to-chip liquid because their product heat splits and airflow and inlet limits leave no air-only path. That conclusion holds for the specific equipment and facility, not as a universal rack-kW threshold. The question ahead is what follows single-phase D2C when per-die thermal density keeps climbing toward and past 2 kW.

The roadmap forks into two candidate successors, shaped as much by chemistry and liability as by thermodynamics. Two-phase approaches (two-phase D2C and two-phase immersion) offer better heat transfer by exploiting the latent heat of vaporization. But two-phase immersion stalled when the PFAS health-and-liability crisis drove 3M to exit the Novec fluorochemical business, leaving the supply chain and regulation of dielectric fluids deeply uncertain. Microfluidics etches coolant channels directly into the silicon, as in Microsoft's research demonstrating up to ~3x cold-plate performance with AI-designed bio-inspired channels. It is the more promising far-term path because it attacks thermal resistance at its source, the die-to-coolant interface, rather than downstream of it. Neither is a 2026 production decision, so the conservative move is concrete: build for single-phase D2C now, but reserve the mechanical and facility-water headroom (CDU capacity, secondary-loop delta-T margin, manifold and quick-disconnect provisioning) so the hall keeps an adoption option. The successor's coolant chemistry, pressure, materials, water flow and service procedure must still pass a new qualification before anyone promises to avoid re-plumbing. → DLC in Chapter 5.4; immersion and the PFAS problem in Chapter 5.5; facility water loops in Chapter 5.7.

The accelerator roadmap sets the clock for every other subsystem in the building. NVIDIA's stated cadence is yearly. Vera Rubin (NVL72; Pegatron rates it 188 kW Max Q / 228 kW Max P) has been shipping since June 2026. NVIDIA targets Rubin Ultra for H2 2027, either as the NVL576 system (eight MGX NVL72-class racks in one 576-GPU optical NVLink domain) or in the ~600 kW Kyber rack. Kyber NVL144 holds 144 packages. On the quad-die plan NVIDIA announced at GTC 2025, that is 576 compute dies and ~15 ExaFLOPS FP4 inference / 5 ExaFLOPS FP8 training per rack. SemiAnalysis reported on 30 June 2026 that each package has two compute dies, which would make it 288, and NVIDIA has not confirmed either count. The date is contested too. SemiAnalysis reported in July 2026 that Kyber NVL144 slips to 2028 on PCB-midplane manufacturability; NVIDIA says "our roadmap is intact" and still targets H2 2027. Feynman follows in 2028 (Rosa CPU, NVLink 8), and NVIDIA presents eight linked Kyber racks, NVL1152, as its scale-up path. NVIDIA has published no Feynman rack power. NVIDIA's 800 VDC architecture has roadmap headroom toward ~1 MW racks, and OCP's Mt. Diablo 1.0.0 rates one power rack at 800 kW to over 1 MW.

Memory climbs in step. Counted per GPU package, HBM goes H100 80 GB → B200 SXM 180 GB → B300 288 GB → Rubin 288 GB HBM4 (shipping 2026) → Rubin Ultra, announced at 1 TB of HBM4E across a four-die package. In August 2026 The Information, BofA and Mizuho reported NVIDIA testing lower-memory Rubin Ultra configurations during an HBM4E supply squeeze. According to The Information, the versions under test carried 192–256 GB per package; some used HBM4 rather than HBM4E, and some had fewer than the 16 announced stacks. BofA reads the cut as temporary and supply-driven, not a demand peak. In September 2026 SemiAnalysis put Rubin Ultra at 192 GB of 8-high HBM4 per GPU. NVIDIA has confirmed none of these, so Rubin Ultra's per-package specs are in flux. For a strategist the takeaway is the cadence, not the spec sheet. A yearly generation step, set against a contested roughly 2–3-year frontier-economic bear case (the depreciation sensitivity from Chapter 1.8), means the substrate you pour in 2026 must absorb at least two generations of density growth within its first refresh window.

On first live silicon, CoreWeave measured a Vera Rubin NVL72 delivering up to ~10× the tokens per megawatt of a GB200 NVL72 (DeepSeek-R1 at a matched interactivity target). NVIDIA pairs that with one-tenth the inference cost per million tokens, up to 10× the inference per watt, and one-fourth the number of GPUs (~75% fewer) to train a given MoE model, which is a training metric. These are vendor ratios versus Blackwell, not absolute $/token, and no absolute cost per token or tokens per dollar has been published.

The other three suppliers in the screening table below run their own clocks, each with a shipping system and an announced successor. AMD launched the 72-GPU MI455X Helios rack in July 2026. It is in production, with initial systems installed at OpenAI, Meta, Microsoft and Oracle (AMD, 12 October 2026), and MI500 is on AMD's roadmap for 2027. Helios changes more than the tray. It brings a double-wide Open Rack Wide frame, power shelves inside the rack feeding a 50 V busbar, a reference design from AMD and Schneider Electric for racks up to 246 kW, and UALink-over-Ethernet for scale-up. The frame's mass is weighed against NVIDIA's racks further down.

Google's Ironwood (TPU v7) reached general availability on 31 March 2026. Google detailed TPU 8t for training and TPU 8i for serving on 22 April 2026 and says they arrive later in 2026. TPUs are mostly rented through Google Cloud, but since Q2 2026 Google also delivers TPU systems to select customers' own data centers. Google publishes no TPU rack power, so an on-premises TPU buyer must get the facility basis from the offer.

AWS made Trainium3 UltraServers (up to 144 chips) generally available on 2 December 2025, and Amazon expects the first Trainium4 deliveries in 2027. AWS says Trainium4 will support NVLink Fusion and fit NVIDIA MGX racks, which would put a hyperscaler chip on NVIDIA's rack interface. Trainium stays an AWS-operated service, but it does not always sit in AWS's buildings. In an AWS AI Factory the customer provides the space and power and AWS installs and operates the Trainium racks, so the host's floor loading, water and voltage come from AWS's specification. Outside those sites the substrate follows three clocks: NVIDIA's, AMD's and, where a buyer installs TPUs on premises, Google's. → AMD in Chapter 7.3; TPU and Trainium in Chapter 7.4.

The roadmap risk on this axis is the advanced-packaging gate, not the GPU. Above wafer fabrication, supply is limited by TSMC CoWoS capacity and HBM stacking, not transistors. Contract HBM and CoWoS allocation against the delivery schedule; HBM4 is already in high-volume production for Vera Rubin. For a data-center program the consequence is counterintuitive: your delivery schedule is gated less by your capex than by an upstream packaging line you do not control. A scope that assumes GPU availability on the vendor's stated cadence, without an allocation agreement, underwrites a schedule it cannot guarantee. → NVIDIA roadmap in Chapter 7.2; HBM in Chapter 7.6; advanced packaging in Chapter 7.7.

The network roadmap holds the most consequential standards bet. Unlike power or cooling, where physics mostly dictates the answer, the fabric question is partly political: which ecosystem do you tie your accelerator optionality to? Two fabrics are evolving in parallel. Scale-up, the high-bandwidth accelerator domain that may span one rack or several, is the contested ground. NVIDIA's NVLink (1.8 TB/s per GPU on Gen5, 3.0 TB/s on Gen6) comes with NVLink Fusion, which opens the IP to third-party CPUs and XPUs. The open UALink / UALoE path is the alternative. The consortium's 1.0 specification (April 2025) allows up to 1,024 accelerators in a domain, and its Common 2.0 specification (April 2026) is final. UALink-over-Ethernet and native UALink switching are different implementation paths, and a published specification does not prove that two vendors' parts work together. The third path is Ethernet scale-up, now two OCP specifications: ESUN 1.0 for the switch side (effective 2026-02-09) and SUE-T 1.0.0 for the endpoint transport (Broadcom and Arista, effective 2026-10-08; Chapter 8.2). Scale-out, the cluster fabric connecting independent scale-up domains, is consolidating faster around Ultra Ethernet (UEC 1.0, June 2025; current release 1.1, October 2026; Chapter 8.4) as the open answer to InfiniBand, alongside NVIDIA's Spectrum-X. On 31 May 2026 NVIDIA declared the Rubin-generation Spectrum-X switch, the 102.4 Tb/s Spectrum-6, in production, including its CPO version (Spectrum-X Ethernet Photonics). NVIDIA’s product page names CoreWeave, Lambda, Meta, Microsoft and Oracle Cloud Infrastructure as first adopters of the CPO version, with availability in the second half of 2026; no operator has published field failure rates (Chapter 8.10). CPO in the top-of-fabric switch is therefore orderable but not yet field-proven (Chapter 8.4). Broadcom said on its 2 September 2026 earnings call that it had taped out Tomahawk 7, a 200 Tb/s Ethernet switch chip and the next merchant switch generation, but it gave no ship date or lane rate.

The scale-up standard you underwrite is a multi-year lock-in. It sets which accelerators you can rack, how large a scale-up domain you can build (and so your tensor- and expert-parallel ceiling), and your copper-versus-optics reach budget. Choose NVLink and you get the largest, most mature scale-up domains and the deepest software stack, at the price of single-vendor accelerator dependence. Choose UALink or ESUN/SUE-T and you keep multi-vendor optionality, at the price of a younger ecosystem and smaller proven domains. Under both sits the optics transition. As lane rates climb to 200G and 448G and reach budgets shrink (passive copper ~1–2 m at 800G/1.6T), co-packaged optics moves the optical engine onto the switch package. In NVIDIA's August 18, 2025 illustration, the move cuts a 1.6 Tb/s interface from ~30 W pluggable to ~9 W CPO. Chapter 8.10 itemizes that arithmetic beside NVIDIA's separate power claims of 3.5× (March 2025) and 5× (January 2026), and Chapter 8.9 covers pluggable DSP module power. Those are vendor architecture claims, and they do not measure a whole fabric's delivered-work energy. Lambda said in March 2026 that a GB300 NVL72 cluster of more than 10,000 GPUs runs in production on NVIDIA’s Quantum-X Photonics InfiniBand switches (Chapter 8.10). A Kyber rack on its own keeps scale-up on copper. Optics become a critical-path item only under two conditions: you buy an NVLink domain spanning several racks (the eight-rack NVL576, or a Feynman-generation domain with co-packaged optical NVLink), or you choose CPO switches for scale-out. → scale-out standards in Chapter 8.4; topology and oversubscription in Chapter 8.5; CPO and fiber plant in Chapter 8.10.

Storage is the subsystem most often left off a roadmap, and the density ramp changes it more quietly than any other. Four threads point forward. First, the data path is moving off the CPU: GPUDirect Storage and GPU/DPU-initiated I/O let the accelerator pull data over NVMe-oF without a host-CPU bounce, and the new DPUs (NVIDIA BlueField-4 at 800 Gb/s) are built to own this path. Second, the spinning disk is being designed out of the hot path. All-flash is becoming the default for both the training scratch tier and the checkpoint tier, with PCIe 6.0 dense-flash servers (for example 96 E3.S SSDs, ~2.9 PB) emerging to feed it. Third, the historical split between parallel file systems and object stores is converging, as the same platforms (WEKA, VAST, DDN) serve file and object semantics over one flash substrate. Fourth, and most distinctive for inference, CXL and Ethernet-attached flash are becoming a KV-cache tier, a new layer of the memory hierarchy between HBM and bulk storage.

The KV-cache tier carries the sharpest 2026 roadmap signal. As reasoning models emit long decode sequences, the KV cache balloons and holding it in HBM stops making economic sense. The answer is a three-tier hierarchy: HBM, then host/CXL memory, then NVMe and Ethernet-attached flash. NVIDIA is standardizing the hierarchy through CMX, the BlueField-4 context-memory platform, and the NIXL transfer library. Its flash tier is the announced CMX/STX product, a BlueField-4 context-memory tier aimed at the Rubin generation. A SuperPOD-scale complex pencils out at ~9.6 PB of NAND (16 enclosures × 4 BlueField-4 × 150 TB, per Chapter 9.7), which makes enterprise SSD a Rubin-generation BOM line and a source of NAND-spot pressure. NVIDIA promised STX partner platforms for the second half of 2026, and none was reported shipping as of October 2026. The gain from offloading prefix caches to flash depends on the prefix-hit rate and on prefix transfer time against the time-to-first-token budget. For the facility, storage is no longer a back-of-house capacity question. It is part of the inference memory hierarchy and sits on the latency-critical path. A 2026 design that treats storage as bulk capacity, decoupled from the fabric and the accelerator, is built for the training workload of 2023, not the inference workload of 2026. → the CPU-bypass data path in Chapter 9.3; inference and KV-cache storage in Chapter 9.7; object/capacity tier in Chapter 9.6.

The five subsystem roadmaps converge on a chassis, not a server. The defining structural shift of 2026 → 2030 is that the unit of design stops being the 19-inch rack of independent servers and becomes an integrated power-and-cooling enclosure, where compute, NVLink switching, liquid distribution and power delivery are co-engineered as one mechanical object. The HPE GB200 NVL72 (132 kW nominal, with its own mass and power-distribution drawings) is the first generation that can only be understood as a single integrated unit. NVIDIA’s separate October 2024 OCP reference explains its over-5,000-cable copper backplane, blind-mate liquid manifolds and 1,400 A busbar design. GB300 NVL72 shipped in 2025 and is deploying through 2026 alongside GB200. NVIDIA's reference architecture puts a full rack at up to 142 kW, and Lenovo rates it at 135 kW TDP and up to 155 kW peak. VR200 NVL72 has been shipping since June 2026. Pegatron's datasheet rates its operating profiles at 188 kW Max Q / 228 kW Max P, while NVIDIA's DSX reference design sizes the cabinet at 330 kW TDP for power and cooling.

The ~600 kW Kyber rack (NVL144) is the endpoint. At that power the busbar becomes liquid-cooled and power conversion leaves the compute rack for a disaggregated sidecar in an adjacent power rack. Two designs compete for that role. NVIDIA builds Kyber on a unipolar 800 VDC architecture. It said in August 2026 that its MGX-compatible 800 VDC power rack (~660 kW by SemiAnalysis's figure) would arrive in the second half of 2026; as of mid-October 2026 no shipment had been announced. The alternative is OCP's Mt. Diablo (final 1.0.0, October 2026), a bipolar ±400 VDC design the spec now calls differential 800 VDC. Their grounding and protection bases differ (Chapter 4.7, Chapter 4.11). Scale-up inside a Kyber rack stays on copper, over a PCB midplane. SemiAnalysis blames that midplane's manufacturability for the reported 2028 slip. Optical NVLink appears only across racks. The eight-rack NVL576 uses copper inside each rack, and the accounts differ on the links between racks. NVIDIA's Ian Buck has described pluggable optics in the spine. SemiAnalysis describes CPO between the NVSwitches instead, and expects NVL576 to slip or to ship in small volumes. With Feynman, NVIDIA says scale-up itself becomes a choice between copper and co-packaged optical NVLink (Chapter 8.2). Optical I/O on the accelerator package itself arrives on a similar timeline. Marvell, which completed its acquisition of Celestial AI on 2026-02-02 for its Photonic Fabric optical chiplets, expects Celestial AI to start contributing revenue in the second half of its fiscal 2028, which ends in early 2028.

For facility design this collapses several previously independent decisions into one substrate commitment. No vendor mass is published for a ~600 kW chassis, and none should be inferred from kW. Shipping racks already span ~3,000–3,300 lb wet (GB200 and GB300 NVL72), about 4,000 lb for the Vera Rubin NVL72 (NVIDIA), and nearly 7,000 lb for AMD's double-wide Helios (AMD, CES 2026). The chassis implies a structural slab and seismic-anchoring basis that is costly and disruptive to strengthen later. Size it to OCP's Open Data Center specification (rev 0.7.0, effective 2026-08-27). Its 2027–2028 design targets, not measured racks, are 8,000 lb for the heaviest single rack on 28 × 48 in and 15,000 lb for an ML rack plus sidecar on 56 × 48 in, with handling equipment counted in the moving load. That OCP specification measures rows in power per linear foot, not in length: 25 kW/ft blended across the row and 350 kW/ft at peak IT. Its targets for those years also set a floor of 12 MW per cluster and four power blocks per hall (Chapter 6.1; Chapter 6.2, Chapter 6.7). The rest of the commitment is an 800 VDC riser and busway you cannot rip out without a hall outage; facility-water capacity and pipe-rack geometry sized for liquid heat rejection at that density; and a row-and-aisle pitch set by the chassis footprint plus its sidecar. The rule is the same as across the whole roadmap: reserve the irreversible substrate for the chassis you will house in 2027–2028, and keep the reversible fit-out (the actual trays, transceivers, CDUs and storage shelves) matched to the generation you are buying this year. → rack as integration unit in Chapter 7.13; modular/prefab construction in Chapter 6.4.

The documents behind these interfaces revise as fast as the hardware, so freeze revisions, not product names. For every busway, rack or fabric order you release, name the exact document and revision in the purchase specification, whether OCP or a consortium publishes it. ORV3 Base, ORW Base and ORW Meta Design are separate documents, as are Mt. Diablo 1.0.0 and MGX 1.2.0, and a rack built to one is not approved to another. UALink is published as separate Common, DL/PL, Chiplet and Manageability specifications, so name the combination your switch implements and test it on your fabric. Then apply the rack and software acceptance in Chapter 7.13 and Chapter 7.9.