Advanced Packaging & the Integration Substrate

"How fast is this chip?" used to be answered at the transistor: a smaller node, more transistors, a faster clock. For AI silicon that era is functionally over. A single reticle field, the largest area a lithography scanner can pattern in one shot, is about 858 mm² (roughly 26 × 33 mm), and the largest useful monolithic die is already pressed against that limit. An H100-class GPU is essentially a full-reticle die. You cannot make the logic meaningfully bigger by printing it; you have hit the reticle wall.

Everything that has happened to AI accelerator performance since is, at the physical level, a packaging story: how to stitch several reticle-sized fields together, how to bolt a dozen HBM stacks beside the logic, and how to wire it all with enough bandwidth that the assembled system behaves like one chip. The package is the integration substrate, and the firm that controls the largest, highest-yielding package controls how much compute the world can ship.

This chapter covers the engineering of that substrate: the 2.5D/3D packaging taxonomy and why it sits beside leading-edge logic wafers, HBM and package substrates as one of the upstream gates on how many accelerators ship; the CoWoS-S/R/L families and SoIC / hybrid bonding, with the reticle-stitching and package-area ceilings each imposes; the interposer fork (silicon, RDL or glass) and its reach, cost and yield triad; how HBM stack count per package falls out of interposer area, the engineering driver behind the allocation view of Chapter 7.6; chiplet disaggregation and UCIe and the economics of every die-to-die crossing; and the thermal, warpage and yield consequences of building packages the size of a coaster. Procurement and allocation of CoWoS capacity stay in Chapter 7.6 and Chapter 2.3; here the concern is the physics.

The constraint moved upstream the same way power did in the facility. The industry assumed the gate on accelerator supply was wafer-out logic: how many GPUs a leading-edge fab could print. Logic still matters, but the gate that bit hardest was advanced packaging capacity, specifically TSMC’s CoWoS lines, which through 2026 are fully booked across both the CoWoS-S and CoWoS-L families. HBM supply gates the same package from the memory side, and neither clears without the other (Chapter 7.6).

Reported allocation for the year runs to roughly a million wafers: NVIDIA alone ~595,000, about 60%, with Broadcom ~15% and AMD ~11% (Tom’s Hardware, May 2026). Monthly TSMC capacity climbs from ~75–80k wafers toward ~120–140k by year-end, plus ~50–60k at OSAT partners (TrendForce, June 2026). Institutional investors quoted by Economic Daily News in the same report expect the CoWoS supply-demand gap to narrow from about 20% in mid-2026 to roughly 10% by year-end: still tight, but no longer a wall. Estimates for the next two years differ: about 160–200k wafers a month by the end of 2027 (KGI says 170k, March 2026; July 2026 supply-chain reports say 200k or more) and about 260k by the end of 2028 (Economic Daily News, via TrendForce, September 2026). TSMC itself states only relative growth, more than 80% a year for CoWoS/SoIC capacity through 2027, so every absolute figure is an analyst estimate and none is this buyer’s monthly allocation.

A logic wafer that cannot be packaged is not an accelerator. A TSMC/CoWoS design can hold N3/N5 logic allocation yet ship late if its package slot or qualified HBM lot is missing, and a total wafer count cannot be converted into delivered GPUs without package area, yield, test and product mix.

Under every CoWoS interposer sits an organic package substrate, ABF build-up layers on a glass-fibre core, and in 2026 that layer ran short too. Large AI substrates use low-CTE T-glass cloth to hold warpage down, and Nittobo, the dominant supplier, has said its new lines will not close the gap with demand (heise, February 2026). Counterpoint (March 2026) forecast substrate price rises of 3–5% a quarter through the first half of 2026, and said new greenfield capacity would not come online until 2027. A CoWoS slot without a qualified substrate lot does not ship. Ask your integrator which substrate maker and glass-cloth source the package is qualified on, and treat a change of either as a requalification.

A more aggressive package, with more stitched reticles and more HBM stacks, buys performance. It also consumes more interposer area per accelerator (a 5.5×-reticle part eats far more interposer area, and yield risk, than a 2× part) and can put the program onto a newer process with less production-yield history. A conservative package preserves yield and supply but caps memory bandwidth and compute area. No axis is free: the package is where the silicon team’s performance ambition meets the packaging line’s yield and capacity limits.

Advanced packaging splits cleanly into two regimes, and the fork between them is the first decision an architecture team makes. 2.5D places multiple dies side by side on a shared horizontal carrier, an interposer, that routes thousands of fine wires between them. This is how logic meets HBM today: the GPU die and its HBM stacks sit beside each other on a silicon or RDL interposer, and HBM's wide thousand-plus-bit bus fans out across that carrier. CoWoS is the canonical 2.5D flow.

3D stacks active dies vertically, face to face and bonded directly, so a die sits on top of another die rather than beside it. TSMC's SoIC and the hybrid-bonding processes underneath it are the 3D flow. AMD's MI300 stacks compute chiplets on base dies this way, and the HBM stack itself is a 3D structure internally.

The consequence of the fork is geometric. 2.5D buys area: you spread components across a large flat carrier, and the limit is how large an interposer you can yield. 3D buys proximity and density: wires between stacked dies are microns long instead of millimeters, which slashes interconnect energy and latency. The price is a severe thermal problem and a yield-multiplication problem. With a top-mounted cold plate, a buried lower die must send heat through the overlying material toward the spreader and plate, and a bad die anywhere in the stack can scrap the whole assembly. Some 2026 accelerators are hybrids: a 2.5D interposer carrying 3D-stacked logic and 3D-stacked HBM. The package is two integration technologies at once.

CoWoS-S uses one big slab of TSV-bearing silicon as the interposer. It gives dense silicon routing, but a large silicon interposer is expensive and its own reticle-stitching limits cap its size. CoWoS-R drops the silicon for an organic RDL carrier, with different routing, warpage and assembly constraints; price, yield and enabled stack count depend on the flow you select. CoWoS-L is the 2026 frontier. It abandons the single monolithic silicon interposer for an RDL carrier with small local silicon interconnect bridges embedded only where dense die-to-die routing is needed. The package can then span far more than a single reticle while spending silicon only where it justifies its cost. TSMC's roadmap runs 3.5× in production since 2024, 5.5× in volume production in 2026 at a reported yield above 98%, 9.5× targeted for 2027 with 12+ HBM4-generation stacks (2025 symposium), and 14× with 20 HBM stacks targeted for 2028.

Choosing -L means inheriting new yield-loss mechanisms: bridge-to-RDL alignment, stitching seams and larger-package warpage. In exchange you get a larger qualified routing area. Compare usable routes and package yield, not the family suffix.

Where 2.5D spreads outward, SoIC (System-on-Integrated-Chips) stacks upward using hybrid bonding, a bumpless, copper-to-copper, dielectric-to-dielectric direct bond that eliminates the solder microbumps of older 3D stacking. The payoff is interconnect density. Classic solder microbump pitch is ~20–50 µm; hybrid bonding takes the bond pitch into the single-digit-micron range (TSMC SoIC: 6 µm, with 4.5 µm on its A14 roadmap), and sub-micron pitches have been shown in wafer-to-wafer research. That is one to two orders of magnitude more vertical connections per unit area, which lets two dies behave electrically as if they were one: picojoule-per-bit interconnect, full-bandwidth die-to-die links, latency measured in fractions of a clock.

AMD's MI300-class parts stack compute chiplets on a base die this way. HBM's internal joining process is a separate question, and it has not settled in hybrid bonding's favour: SK hynix announced 16-layer HBM3E in 2024 and shipped 12-layer, 48 GB HBM4E samples in June 2026, both on Advanced MR-MUF. Treat stack height and joining process as independent attributes, per supplier and per product.

The 3D ceiling is thermal, and it is a hard one. When you put logic on top of logic, one die is always buried. With the cold plate above the stack, it is the lower die whose heat must traverse the upper die to escape, while power density at the bond interface can exceed what any external cooling can extract without throttling. Draw the cooler, the die order and the heat path for the actual assembly before naming the bottleneck: AMD puts second-generation 3D V-Cache below the CPU cores precisely to keep the hotter logic facing the cooler. So 3D-on-logic stays selective. You stack the parts that benefit most from proximity (cache, certain compute tiles) and keep the hottest, highest-power logic where it can see the cold plate directly. Going 3D makes the cooling problem of Part 5 harder in exchange for interconnect you cannot get any other way. Large-package hotspot and heat-flux behavior is engineered in Chapter 5.1 (the density wall) and removed by the cold plate in Chapter 5.4.

Underneath the CoWoS branding sits the real material decision: what is the carrier made of? Separate the interposer or redistribution layer from the package substrate core before comparing reach, cost and yield.

Silicon interposers route the finest lines and carry TSVs for power and signal straight through, which gives dense routing for HBM's wide bus. But a large silicon interposer is costly, is itself reticle-limited (you cannot pattern one bigger than a scanner field without stitching), and its CTE mismatch against the organic substrate drives warpage as it grows. RDL (redistribution-layer) carriers can be made large, but their routes are coarser than local silicon interconnect: CoWoS-L uses silicon bridges where the RDL carrier alone cannot supply the required density. Cost and warpage depend on the complete stack-up, not automatically on the carrier material.

Glass is the emerging third option. Glass-core package substrates and glass interposers are separate structures, and proposals for each aim at silicon-like dimensional stability and flatness, with potential panel-scale manufacturing and excellent high-frequency loss characteristics. The ecosystem is immature, though: handling, via formation and crack-propagation risks are still being industrialized through 2026.

This is the link between this chapter and the memory chapter, and it is close to an identity: HBM stacks per package is a function of interposer area, not of how much HBM you can buy. Each HBM stack occupies a fixed footprint beside the logic die and must be reached by a bus of a thousand-plus wires fanned across the interposer. Read the platform sizes as limits, not as a law of stack count. TSMC’s ~3.3–3.5×-reticle platforms accommodate eight HBM stacks, and the ~5.5× platform that CoWoS-L is producing in volume in 2026 accommodates up to twelve. TSMC’s published pairings are 12+ stacks at the 9.5× package targeted for 2027 and 20 stacks at the 14× package targeted for 2028. NVIDIA’s announced Rubin Ultra package carries 16 stacks; the warpage section below covers a report (SemiAnalysis, unconfirmed) that it was cut to two dies. Interposer size follows the whole floorplan (logic-die area, I/O and routing as well as stacks), not the HBM generation alone.

So when Chapter 7.6 describes HBM as the largest modeled BOM line and the allocation gate, the engineering reason a given accelerator carries its stack count lives here. The package area the program chose set the stack count, which set the memory capacity and bandwidth, which set where it lands on the memory roadmap.

Memory ambition and packaging ambition are therefore one decision. A team that commits to an 8-stack, ~288 GB HBM4 accelerator (Rubin-class: 19.2 TB/s per GPU at NVIDIA’s shipping spec, about 2.4 TB/s per stack; 22 TB/s was the launch announcement) has by that act committed to a large-reticle CoWoS-L package, its yield curve, and its slice of the scarcest packaging capacity on earth. NVIDIA has not disclosed Rubin’s exact reticle multiple, and eight stacks alone do not require the full 5.5×. You cannot buy your way to more bandwidth without buying your way to more interposer area, and interposer area is one of the things in shortest supply. The procurement reflex, "order more HBM", is necessary but not sufficient: without the CoWoS slot to mount it on, the HBM is inventory, not bandwidth. That allocation logic lives in Chapter 2.3; the engineering driver lives here.

Once the reticle wall makes one big die impossible, the question becomes how to build a big system from small dies. That is the chiplet decision. Disaggregation splits a would-be monolithic SoC into multiple dies ("chiplets") that are packaged together: compute tiles, I/O dies, cache dies, each potentially on a different process node tuned to its job. The upside is real and quantifiable. At a fixed defect density, smaller die area improves individual-die yield, because defect density punishes large dies brutally. Four small dies can therefore improve known-good-die supply, but assembly yield and testing decide the cost of a usable package. You reuse a chiplet across many products. You put I/O on a cheap mature node and spend leading-edge wafers only on the logic that needs them. AMD productized this years ago; it is now the default architecture for the largest accelerators.

The cost of disaggregation is paid at every die-to-die crossing, and this is where UCIe (Universal Chiplet Interconnect Express) enters. A monolithic SoC pays wire, buffering and switching energy on-die. A chiplet system must drive and receive across a physical die boundary, with serialization and encoding determined by its interface. That spends area on PHYs, power on the link (picojoules per bit that add up across terabytes per second) and latency on the crossing. UCIe standardizes that interface to support interoperability within an implemented, qualified profile. UCIe-S targets standard 2D packages, UCIe-A targets advanced 2.5D packages, and UCIe-3D covers 3D integration; 64 GT/s arrived with UCIe 3.0, while the 3D interface was introduced in UCIe 2.0. The consortium's published catalogue lists 3.0 as the current revision, but voltage, bump pitch, lanes, protocol, test access and thermal/mechanical compatibility still need agreement.

The strategic opportunity is an open chiplet market: a buyer can, in principle, assemble an accelerator from best-of-breed dies rather than one vendor's monolith. It is the packaging-era analog of the open-system disaggregation Part 8 describes for the network. OCP's Open Chiplet Economy adds a system layer above the die-to-die link. Its Foundation Chiplet System Architecture (FCSA 1.1.0, Arm and Secure-IC, October 2026) defines ISA-neutral chiplet types and the interfaces between them, and carries their memory traffic over UCIe 1.1 streaming or OCP's Bunch of Wires. Ask a chiplet supplier which FCSA compliance level (0, 1 or full) and which transport its die meets.

Transmitted rate = 1.0×10¹² B/s × 8 bit/B × 1.10 = 8.8 Tb/s. Combined endpoint power for A = rate × 1.0 pJ/bit = 8.8 W; B at 3.0 pJ/bit needs about 26 W and exceeds 20 W. Buried-die rise = 600 W × 0.040 K/W = 24 K, within 30 K. So A passes and B fails on link power. A’s payload limit is 20 W/(8 × 1.10 × 1.0 pJ/bit) ≈ 2.27 TB/s, so A fails at about 2.3 TB/s. Independently, the thermal crossover is 30 K/600 W = 0.050 K/W; at 0.060 K/W the rise is 36 K and A fails.

If A passes, write its actual bump pitch, voltage, lane width, protocol, test access and assembly process into the acceptance test: a UCIe version number alone does not make chiplets interchangeable. Make thermal and assembly-yield results a condition before you reserve HBM placement and packaging capacity.

The interface terms come from the UCIe Consortium specification catalogue; Chapter 7.14 covers integrating and accepting the finished server.

Large packages punish you three ways, and each is a downstream cost of the area you bought to defeat the reticle wall. Warpage comes first. A large silicon interposer on an organic substrate has a coefficient-of-thermal-expansion mismatch, so as the assembly heats and cools it bows. Past a certain span the bow threatens the solder joints to the board and the bond integrity across the package. That is a major reason CoWoS-L's RDL-plus-bridge construction exists (the organic carrier is more compliant) and why glass cores are attractive (dimensionally stable and flat). Warpage is a first-order limit on how large a package you can build at all, not a yield footnote.

The limit is not theoretical. Rubin Ultra, as NVIDIA announced it at GTC 2025, carried 16 HBM4E stacks and four near-reticle dies on a single CoWoS-L package. On June 30, 2026 SemiAnalysis reported that NVIDIA had cut it to two dies per package over manufacturing concerns. Secondary outlets attribute the cut to substrate warpage at four-die scale. NVIDIA has not confirmed the change. Until some product ships on a new package size, that size is a yield risk.

Yield comes second, and large packages multiply it sharply. A 12-HBM, multi-reticle, stitched package is the product of many independent yields (each die, each HBM stack, each bond, each stitch seam), and a defect anywhere late in the flow scraps an assembly carrying enormous accumulated value: a dozen HBM stacks and multiple known-good logic dies. That is why known-good-die testing before assembly is non-negotiable, and why the largest packages carry the largest scrap-cost exposure. You are gambling the most expensive components on the last and least-reversible step.

Thermal comes third. A large, dense package concentrates hundreds of watts into a small footprint, with internal hotspots over the 3D-stacked regions. The resulting heat fluxes are more than air can remove, and even direct-to-chip liquid must be engineered for them. The package's heat-flux map is the boundary condition the cold plate inherits, which is the explicit hand-off to Chapter 5.1 and Chapter 5.4.

Walk the chain forward and the package sits at the center of it. The reticle multiple you choose sets the interposer area. Interposer area sets the HBM stack count, which sets memory capacity and bandwidth (→ Chapter 7.6). The reticle multiple also sets which CoWoS family and OSAT can build the part, which sets yield and lead time (→ Chapter 2.3). Package size also sets the organic substrate under it, a separate supply line with its own materials shortage in 2026. The package's power and heat-flux map sets the cooling boundary condition (→ Chapter 5.1, Chapter 5.4). And the on-package power delivery and di/dt behavior that a big, dense package demands is its own engineering problem (→ Chapter 7.12).

The package is the substrate every other accelerator decision is mounted on. Get its reticle multiple and family wrong and you have either an accelerator you cannot get capacity to build or one whose memory and compute ambition you under-shot. Unlike a board respin, a package family change is a multi-quarter, multi-million-dollar reset.