Hyperscaler XPUs: TPU, Trainium/Inferentia, Maia, MTIA

The previous two chapters covered silicon you can buy: NVIDIA and AMD sell merchant accelerators to anyone with a purchase order. Hyperscaler XPUs you mostly rent, or build. Google's TPU, AWS's Trainium and Inferentia, Microsoft's Maia and Meta's MTIA are not products on a price list. Each is a vertically integrated cost structure: a chip co-designed with a compiler, wired into a proprietary fabric, and deployed in racks the same company owns, serving models that the company or its anchor tenant runs. TrendForce forecasts ASIC-based servers at roughly 27% of 2026 AI-server shipments (TrendForce, April 2026 revision). That is a forecast of server units, not a split of workload, tokens, revenue or energy.

Most operators do not select an XPU; they inherit one when they pick a provider. TPUs are mostly rented through Google Cloud, so you rent a Cloud TPU slice and accept XLA. Trainium means renting a Trn instance and accepting the Neuron SDK. The decisions here are therefore second-order: stable workload against portability, captive against merchant, renting an anchor tenant's economics against building your own. Underneath all of them runs one structural trend: the model owner concluding that the cheapest way to serve its own tokens is to design the chip that serves them.

Raw FLOPS alone do not decide this: NVIDIA spreads its R&D across merchant volume, while Google and AWS can optimize an XPU around their own workloads. Three pressures push a model owner to build. First, cost per token at volume. If you serve hundreds of billions of tokens a day on a model architecture you control and rarely change, a workload-specialized XPU that stays programmable through a compiler such as XLA or Neuron can beat a general-purpose GPU on tokens/$/W when the workload lowers cleanly. At hyperscale, single-digit-percent efficiency gains are nine-figure annual line items. Second, supply independence. NVIDIA allocation can be the limiting factor for a specific delivery window (see Chapter 7.6 on HBM and Chapter 2.3 on long-lead procurement). A hyperscaler with its own tape-out and its own TSMC wafer and CoWoS allocation is not standing in NVIDIA's queue. Third, the NVIDIA margin. Avoiding NVIDIA's purchase price means taking on design, software, yield and sustaining costs, so NVIDIA's gross margin does not turn one-for-one into program savings.

All three pressures need scale to pay off, which is why most operators should not build. Custom silicon carries a program-specific NRE and development schedule, a hard minimum volume below which per-unit cost is worse than renting GPUs, and a permanent software-engineering tax to keep the compiler competitive (Chapter 7.5 has the ASIC break-even math). An entity with a durable workload, committed volume and a funded design and software organization can test that bar. An external renter tests the instance, region and reservation on offer instead. Internal-only silicon is not rentable capacity.

The TPU is the only hyperscaler XPU with a decade of generations behind it, and it shows that the model-owner-builds-silicon thesis works at frontier scale: Gemini is trained and served on TPUs end to end. The 2026 flagship is Ironwood (TPU v7), announced in April 2025, previewed in November 2025, made generally available March 31, 2026, and explicitly positioned as 'the first Google TPU for the age of inference.' Each chip delivers 4,614 FP8 TFLOPS and 192 GB of HBM3E at 7.37 TB/s, a 6x memory jump over Trillium (TPU v6) and the largest single-generation memory expansion in TPU history, at roughly 2x the perf/watt of the prior generation (Google Cloud; SemiAnalysis, Nov 2025).

Ironwood is what you can rent; the eighth generation is announced, which is not the same thing. On April 22, 2026 Google detailed TPU 8t, a pre-training part with a 9,600-chip 3D-torus superpod, SparseCore embedding offload, native FP4 and the two-layer Virgo scale-out fabric. It also detailed TPU 8i, a sampling-and-serving part with 3x Ironwood's on-chip SRAM, a Collectives Acceleration Engine that replaces Ironwood's SparseCores, and a high-radix Boardfly ICI topology that holds a chip-to-chip diameter of seven hops where a 1,024-chip torus needs sixteen. Per chip, TPU 8t delivers 12.6 PFLOPS of FP4 with 216 GB of HBM at 6.5 TB/s and 128 MB of on-chip SRAM. TPU 8i delivers 10.1 PFLOPS of FP4 with 288 GB at 8.6 TB/s and 384 MB of SRAM, so 8i trades some compute for memory (Google, April 2026; Google does not say whether the FP4 peaks are dense or sparse). Google claims up to 2x perf/watt over Ironwood for both. Broadcom said on September 2, 2026 that it had begun production shipments of TPU 8i to Google, but as of October 2026 neither part is generally available on Google Cloud (Google: later in 2026). Read all of this as announced architecture, not schedulable capacity: the training and serving split changes the shape of the pod you will eventually buy, not the one you can buy now.

What separates TPU from every GPU fabric is the interconnect. TPUs are wired as a 3D torus over the Inter-Chip Interconnect (ICI, 9.6 Tb/s per chip on Ironwood), and the torus is stitched together by optical circuit switches (OCS), Google's own MEMS-mirror switches that physically reconfigure which chips are neighbors. An Ironwood pod scales to 9,216 chips delivering 42.5 FP8 ExaFLOPS with about 1.77 PB of aggregate HBM. This is how Google runs 9,216-chip ICI domains with distributed HBM and software-managed communication, without the all-to-all switch silicon that dominates a comparable GPU cluster: the optics are the scale-up fabric. → Chapter 8.2 treats OCS-vs-switched scale-up in full.

The OCS decouples the logical topology a job sees from the physical wiring. That buys per-job topology on demand (a job requests a torus shape and the OCS provisions it) and lets Google build a slice from healthy chips around unavailable ones. It does not by itself keep a running job alive. In ordinary multi-host Cloud TPU use, a failed instance or ICI link invalidates the mapped topology, and the whole slice is recreated or rescheduled: automatically in managed capacity mode, by the customer in All Capacity mode. Restart-free progress is a separate, framework-level feature that Google attributes to JAX and Pathways.

The lock-in runs deep. There is no CUDA on a TPU. On Ironwood the supported frameworks are JAX and PyTorch, both lowered through XLA, the compiler that turns a high-level graph into TPU machine code; Google's TPU7x documentation says TensorFlow is not supported (older generations supported it). Native PyTorch through TorchTPU, with eager mode, has been in preview with select customers since April 2026. XLA is excellent for the dense and MoE transformer shapes Google cares about, but it is a compiler-first model, not a kernel-first one. You express the computation and trust XLA to schedule it. Where the compiler's choice is not good enough, you drop into Pallas, JAX's kernel language for hand-written TPU kernels with explicit control over blocking and memory movement. You do not get a PTX-equivalent assembly layer or CUDA's two decades of third-party tuned kernels, so the honest comparison is supported ops, compiler constraints, profiling and performance portability, not custom kernels versus none. For a stable architecture the compiler is a feature, because it optimizes across the whole graph. For a research workload chasing a new operator every week it is friction.

TPUs are mostly rented through Google Cloud; since Q2 2026 Google also delivers TPU systems to select customers' own data centers. Alphabet began recognizing revenue from those sales in Q2 and expects most of the contracted revenue in 2027. Anthropic ordered Ironwood systems through Broadcom (disclosed December 2025). The Blackstone–Google TPU venture, announced May 18, 2026, aims to bring its first capacity outside Google Cloud online in 2027. All of these routes keep you in the TPU ecosystem. Leaving it means validating another accelerator's backend, sharding and custom kernels, and the effort depends on your workload. → Chapter 7.9 quantifies the XLA-vs-CUDA switching cost.

AWS built two chips, Inferentia for inference and Trainium for training, and now pitches Trainium for both. The two share one software stack, the Neuron SDK, which sits under PyTorch and JAX. The 2026 story is Trainium3, announced at re:Invent 2025. Each Trainium3 chip delivers 2.52 FP8 PFLOPS with 144 GB of HBM3E at 4.9 TB/s, and it is the first 3nm AI accelerator AWS has shipped. The system unit is the Trn3 UltraServer: up to 144 chips for 362 FP8 PFLOPS, 20.7 TB of HBM, and 706 TB/s of aggregate bandwidth, with about 4.4x the performance and 4x the perf/watt of the Trn2 UltraServer (AWS, Dec 2025). The scale-up fabric is NeuronLink-v4 with a NeuronSwitch-v1 all-to-all topology at 2 TB/s per chip: AWS builds a vertical scale-up like NVIDIA's, against an open-Ethernet scale-out (EFA).

Both TPU and Trainium now run on the anchor-tenant model. In April 2026 Anthropic signed with AWS for up to 5 GW of new capacity, on Graviton and on Trainium2 through Trainium4. OpenAI is a second Trainium anchor, committing about 2 GW through AWS beginning in 2027 (Amazon, 2026). On the TPU side, Anthropic signed with Google and Broadcom in April 2026 for multiple gigawatts of next-generation TPU capacity from 2027, and put it at 5 GW in its May 28, 2026 funding announcement. On September 2, 2026, Broadcom's CEO gave Broadcom's own outlook: it expects Anthropic to bring 5 GW of TPU 8i into service in 2027, and it has line of sight to a further 10 GW in 2028. Project Rainier, the Trainium2 cluster, went from announcement (re:Invent, Dec 2024) to a live cluster of ~500,000 Trainium2 chips in under twelve months. AWS describes it as spread across multiple US data centers, including the $11B AWS campus in St. Joseph County, Indiana. In April 2026 Anthropic reported that it uses over one million Trainium2 chips to train and serve Claude (AWS; Anthropic, 2026). AWS reports that Trainium customers, among them Ricoh, Karakuri and Splash Music, cut training and inference costs by up to 50%, with no baseline given (AWS, December 2025).

A large anchor commitment reduces demand risk and spreads program cost over committed volume, but it does not show that the program pays back; Chapter 7.5 covers break-even. By mid-2026 the chip business was a headline in its own right. Amazon's Q2 release (July 30, 2026) put the chips business at an annual revenue run rate above $25B. That figure is not Trainium-only: Amazon names Trainium and Graviton as the chip businesses, with Nitro as their infrastructure. Andy Jassy put Trainium revenue commitments above $225B on the Q1 2026 earnings call. Trainium4 is still a roadmap item with no firm shipping date from AWS. When Amazon announced its OpenAI partnership in February 2026, it said it expected the first Trainium4 deliveries in 2027. The anchor is not exclusive. Anthropic runs Claude on Trainium, TPU and NVIDIA GPUs, and it has gone to AMD as well. Its July 2026 agreement covers up to 2 GW of Helios capacity on the MI450 series, and deployment is planned to begin in H1 2027 (Chapter 7.3). For everyone else, the consequence is that you are renting into economics sized for someone else's model, on a Neuron stack whose roadmap follows the anchor's needs, not yours.

The Access model column governs strategy. Trainium and Inferentia are captive to AWS, Maia and MTIA are not generally rentable, and TPU is mostly Google Cloud plus select customer-site deliveries. AWS AI Factories put Trainium in customer buildings, but AWS operates it. Each chip comes from one supplier, so choosing an XPU is downstream of choosing a provider, and choosing a provider is downstream of choosing a software stack you are willing to live inside. The perf/watt numbers matter, but what you are really picking is an ecosystem.

The third cohort is the newest, and the clearest signal of where the industry is heading. Microsoft Maia 200, announced in January 2026, uses TSMC 3nm, 140B+ transistors, >10 PFLOPS FP4 and >5 PFLOPS FP8, and 216 GB HBM3E at 7 TB/s in a 750 W package. Microsoft claims about 30% better performance per dollar than the best hardware in its own fleet (vendor claim). Maia 200 ships with a co-designed system: a second-generation closed-loop liquid-cooling heat-exchanger unit (HXU) that rejects heat through an integrated exchanger, so Azure can install it in existing air-cooled halls without re-plumbing the building (a deliberate hedge for the density ramp; see Chapter 5.10 on liquid retrofits), and a two-tier Ethernet scale-up network of up to 6,144 accelerators. Microsoft's October 12, 2026 update says Maia 200 now runs production workloads, including Microsoft 365 Copilot and OpenAI, MAI and open-source models, and is rolling out across the US, Europe and Australia. Microsoft plans to bring GitHub Copilot and Microsoft Foundry onto Maia 200 by the end of 2026.

The successor, Maia 300, was disclosed the same day, October 12, 2026, with production deployments planned from 2027. Microsoft's announced specifications put 96 accelerators in a 167 kW rack, each with 288 GB of HBM4 at 18 TB/s, and link up to 128 racks into a 12,288-accelerator superpod. Liquid removes more than 80% of the rack's heat, using facility water or an exchanger unit that rejects heat to an air-cooled hall. Each accelerator has a 950 W TDP, with compute chiplets on TSMC N2P. These are announced specifications, not deployed capacity. Microsoft says Maia will scale to multi-gigawatt deployments from 2027 as the primary inference system for its own products and services. It projects up to 7x Maia 200's rack-scale token throughput in a full 96-accelerator rack, and 40% more tokens per megawatt than leading GPU-based systems (vendor claims; Microsoft, 2026-10-12).

Meta MTIA has the most aggressive roadmap. On March 11, 2026 Meta disclosed four generations: MTIA 300, already in production for ranking and recommendation training; MTIA 400, tested in the lab and heading to deployment for GenAI; and the inference-focused MTIA 450 (early 2027) and MTIA 500 (2027). The chips use chiplets with CoWoS packaging, and Meta's August 2026 engineering post gives the MTIA 300 216 GB of HBM3E. The silicon is built for Meta's own workloads and is never sold to anyone.

The most consequential entrant is not a cloud at all. OpenAI taped out its first custom inference chip, 'Jalapeño', nine months after starting the design, and unveiled it on June 24, 2026 with engineering samples running in the lab. OpenAI presented results at Hot Chips on August 25, 2026: on SemiAnalysis InferenceX, Jalapeño beat GB200/GB300 systems by 1.5–1.9x on throughput per watt and cut end-to-end latency by 1.7–3.6x. Jalapeño is rated at 700 W and drew no more than 550 W in those runs; a GB200 GPU is rated at 1,200 W and a GB300 GPU at 1,400 W. These are OpenAI's own results, not independent measurements. Jalapeño also carries HBM4, so the fair comparison is NVIDIA's HBM4 generation, Vera Rubin, not Blackwell.

Jalapeño belongs to a 10 GW collaboration with Broadcom announced on October 13, 2025, whose rack-deployment target runs from H2 2026 through 2029; no financial terms were disclosed. The companies target initial deployment by the end of 2026 (Broadcom and OpenAI announcements), and on September 2, 2026 Broadcom's CEO said Jalapeño is on track for 1.3 GW of deployment in 2027. This is the model-owner-builds-silicon trend in its purest form: a model owner deciding that the cheapest tokens are the ones running on a chip it designed. It also shows the quiet kingmaker. Broadcom is the design and SerDes partner behind Jalapeño, with Marvell playing a similar role elsewhere. The merchant-silicon disruption comes from a handful of model owners renting Broadcom's and Marvell's packaging, SerDes and physical-design expertise to turn workload knowledge into captive silicon, rather than from a swarm of independent chip startups. → Chapter 7.5 on the merchant-silicon ASIC economics; Chapter 8.3 on the SerDes and switch-ASIC supply chain underneath all of it.

For an operator who is not a frontier model owner, the practical question collapses to three options. The right one depends on workload stability and your willingness to enter a single vendor's gravity well.

Stay on merchant GPUs (NVIDIA or AMD). This suits workloads that churn architecture, need supported portable paths, or depend on a kernel ecosystem. You pay your chosen vendor's margin and stand in its allocation queue. NVIDIA gives you CUDA velocity and AMD gives you ROCm, and moving between them still requires a tested port. → Chapter 7.2, Chapter 7.3.

Rent an XPU through its provider. This is the right call when you have a stable high-volume inference workload and the XPU's tokens/$/W beats GPUs by enough to repay the port: a TPU slice for a JAX-native serving stack, a Trn/Inf instance for a PyTorch model that lowers cleanly through Neuron. The consequence is provider lock-in: your serving stack assumes that vendor's compiler, fabric and roadmap, and your exit is a port to another backend. → Chapter 7.9, Chapter 7.11.

Build your own. Reserve this for entities that own the model, ship at hyperscale, and can clear the program-specific NRE, the development schedule and the permanent compiler-team headcount, almost always through a Broadcom/Marvell partnership rather than from scratch. The potential payoff is a separately contracted allocation and a lower cost per token after program costs. The risk is that a workload-specialized design locks you to a compiler and to a model architecture that may move under you. → Chapter 7.5.