CPO, Fiber Plant & Structured Cabling

By this point in Part 8 the fabric topology is decided, the switch silicon is chosen, and the link budget is understood (Chapter 8.9). What remains is the most physical layer of all: the glass that carries the photons, the connectors that join it, and the increasingly fraught question of where the laser lives. An AI back-end fabric at 100k-GPU scale is a cabling project of tens of thousands of fiber-miles and millions of mated connections. At 800G and 1.6T per link, the optics that drive it can consume enough power and cost to justify redesigning the switch around them. Count optical ends, host conditioning and cooling together: moving the engine can shrink one line of the bill while making a failed part harder to isolate and replace.

Three coupled choices run through this chapter. Co-packaged optics (CPO) puts the optical engine on the switch package; the chapter covers the power and signal-integrity economics that justify it and the serviceability tax it imposes. The fiber plant sets single-mode versus multimode, the reach classes, and the channel architecture the whole ramp is trunked against. Structured cabling at scale (MPO trunking, base-N systems, polarity) lives or dies on the loss budget: an install either lands inside it or fails acceptance. Chapter 16.2 consolidates the CPO roadmap to 2030; Chapter 7.15 covers install velocity and field execution.

The case for CPO is a chain of physical facts, and each gets worse as lane rates climb from 100G to 200G to 400G per lane. A signal from a switch ASIC to a faceplate pluggable crosses package balls, PCB traces, a connector and the module's own substrate, and the insertion loss of that path grows steeply with frequency. At a higher lane mode, host-board insertion loss, return loss and crosstalk must still fit the channel budget. A retiming DSP equalizes and regenerates the signal at the cost of active power. FEC corrects errors at its specified host/module boundary and is a separate function. Shortening the electrical distance can reduce the conditioning burden.

CPO attacks the path itself. When the optical engine sits beside the switch ASIC on the same package substrate, the electrical run shrinks from tens of centimeters across a board to millimeters across a package. That can reduce the equalization needed to recover the signal, though the saving depends on which DSP or retiming functions the implementation removes.

NVIDIA’s March 2025 silicon-photonics announcement put its comparison with pluggables at 3.5× power efficiency, 63× signal integrity, 10× network resiliency and 4× fewer lasers. NVIDIA’s August 2025 technical blog breaks the power claim down for one 1.6 Tb/s interface: about 30 W pluggable (20 W DSP plus 10 W laser) against about 9 W co-packaged (7 W optical engine plus 2 W laser). Those parts divide to about 3.3×, though NVIDIA labels the figure 3.5×. NVIDIA’s January 2026 technical blog restates the power figure as 5× lower per 1.6 Tb/s port against pluggable interconnects, and its 2026-05-31 production release repeats the 5× and adds 5× longer AI uptime against networks using traditional transceivers. NVIDIA has itemized only the 3.5×; before the 5× goes into a power plan, ask the supplier for its component breakdown.

Broadcom’s October 2025 Davisson announcement puts optical-interconnect power about 70% lower. These are vendor comparisons with different boundaries, not interchangeable installed-system guarantees. At fleet scale a measured optical saving can return megawatts to compute, but count package integration, fiber attachment, host conditioning, heat removal and diagnostics together.

On 2026-05-31 NVIDIA said Spectrum-X Ethernet Photonics was in production, and in July 2026 TrendForce reported it shipping to select partners. NVIDIA’s product page names CoreWeave, Lambda, Meta, Microsoft and Oracle Cloud Infrastructure as first adopters of Spectrum-X Photonics, with availability in the second half of 2026. NVIDIA’s InfiniBand CPO switch reached the field first: in March 2026 Lambda said a GB300 NVL72 cluster of more than 10,000 GPUs on Quantum-X Photonics switches was running in production. Lambda describes it as one of the largest CPO switch deployments so far; no operator has yet published field failure rates for CPO at that scale.

On the Broadcom side, the previous-generation 51.2T Bailly CPO switch can be bought through system makers today. Partners showed systems built on the 102.4T Davisson at OCP in October 2026. One of them, Micas, plans to start producing its Davisson switch in Q1 2027 (see the key numbers below). The installed base is still small: TrendForce’s March 2026 forecast puts CPO at about 0.5% of the optical modules used in AI data centers in 2026. Before committing to that package boundary, ask for the supported FRU list, the repair duration, the installed power and a delivery commitment.

Radix and lane rate together make the trade most visible at heavily populated fabric tiers: 102.4 Tb/s can be presented as 64 logical 1.6T ports or 128 logical 800G ports when the ASIC and system support those modes. Every populated electrical attachment and optical end adds power and cable obstruction, so compare the complete switch at its required inlet condition and permitted cooling fault. Chapter 8.3 covers its thermal limits, Chapter 8.9 channel testing, and Chapter 16.2 dated supplier status. CPO belongs at the top of the fabric where measured system power and demonstrated restoration justify its integration cost; future radix or module market share cannot settle that for a site.

Read the table as two separate decisions: signal conditioning and service access. A faceplate alternate that has passed your host test can simplify inventory, but it is not interchangeable merely because its cage fits. LPO and LRO keep a faceplate module and change the host’s electrical duties; either can remain the right architecture when its complete channel passes. Near-package and co-packaged engines shorten the electrical route, but a failed internal engine may not be repairable with a faceplate spare and a screwdriver. Open CPX Specification 1.0 (Open CPX MSA, approved 16 September 2026) defines a middle ground: a socket soldered to the ASIC package or the board next to it that takes a 6.4 or 7.2 Tb/s optical module, with internal or external laser, or a copper cable. Its modules are not expected to be hot-pluggable, so ask whether a swap powers down one socket or the whole switch. Optical attach and internal access become part of what the system supplier must support. Nothing forces a march down the rows: choose the combination whose demonstrated power, failure containment and replacement procedure fit your constraints.

Lasers and optical engines fail, and the failed component determines what you must drain to replace it. A faceplate module can be removed without opening the switch, but every link it carries goes with it, including a breakout assembly’s links. An internal optical-engine fault can instead mean pulling a subassembly or a whole board: lose the affected ports, re-cable, restart and requalify. Time the full route from alarm and diagnosis through isolation, access, replacement and loaded acceptance, because insertion time is not restoration time. If the surviving fabric cannot carry admitted work through that procedure, the power saving buys a workload outage.

Ask each supplier for measurements at the boundary you are buying. The first whole-switch power result comes from Micas: roughly 38% less switch power for its 51.2T Bailly CPO box than for the pluggable configuration it tested, in one Supermicro/AMD design (vendor-reported, September 2026). Meta supplies the deepest reliability data. Its ECOC 2025 paper compared Broadcom’s 51.2T CPO switches against pluggable controls in a full-system lab evaluation, and a Meta engineer later said about 15 million device-hours had produced no unserviceable CPO failures. Broadcom also reported over 1 million 400G-equivalent port-hours with no link flap in high-temperature testing at Meta. None of this is a failure rate for your configuration: it is lab and early-deployment data from one operator. Ask for field removals per million device-hours, separately for engines and lasers, along with the failure definition and the temperature profile.

An External Laser Small Form-factor Pluggable (ELSFP) module keeps a serviceable light source at the front panel and feeds light over fiber to the engine that modulates and detects the signal. The OIF ELSFP agreement (linked below) defines a replaceable external-laser interface, and Broadcom’s Davisson record also names field-replaceable laser modules. Neither shows that an engine or fiber attach can be changed live. On NVIDIA’s Quantum-X Photonics Q3450-LD, for example, each of 18 removable light-source modules feeds eight of the 144 MPO ports (Lambda engineering note, June 2026), so ask what one source failure takes down and whether a redundant laser covers it. Ask the supplier which lasers feed which engines and ports, the permitted isolation sequence, the internal assembly FRU, the board fallback and the requalification procedure. For a detachable optical subassembly, check the exact system and service manual instead of carrying the label over from another platform. Put spares, RMA ownership, contamination control and technician access in the purchase contract, because the optical integration is a system dependency: you are buying a supported system, not a replacement optic from a competitive module market.

Laser supply also sets the delivery date, for pluggables and CPO alike. Single-mode 800G and 1.6T pluggable modules use indium-phosphide electro-absorption modulated lasers (EMLs) or continuous-wave (CW) lasers driving silicon-photonics modulators, and CPO switches take CW lasers in their external light-source modules. On its September 2026 earnings call Broadcom said EML and CW laser supply was well short of demand and that it was more than tripling its own laser capacity. TrendForce, forecasting a 2026 AI optical-transceiver market of about $26B (April 2026), named EML and CW laser chips as the tight supply. For a 2027 build, get a dated module or laser allocation into the purchase order, and confirm that an alternate laser is on the switch supplier’s approved list before you count it as a second source.

Beneath the transceiver decision sits a more durable one: which glass you pull. Fiber outlives several generations of optics, so the density ramp must be designed against it. Get it wrong and the next speed step is a re-pull, not a transceiver swap. The fork is single-mode versus multimode.

Multimode (OM3/OM4/OM5) uses a wider core and short-reach VCSEL optics. Modal dispersion, wavelength, fiber grade and connector loss set what it can do. OM4 supports 100GBASE-SR4 to 100 m. In IEEE 802.3df’s 2024 800GBASE-SR8 application, nominal reach stays at 100 m on OM4 and drops to 60 m on OM3, with maximum channel insertion losses of 1.8 dB and 1.7 dB. The shorter 800GBASE-VR8 application reaches 50 m on OM4. These are separate application budgets, not a rule that every speed generation halves reach. Keep installed multimode when the next application works with margin and the endpoint economics favor it. Otherwise a route that used to work becomes a new optical-application purchase or a new fiber pull.

Single-mode (OS2) uses a narrow core and laser-based transceivers. Attenuation and chromatic dispersion govern its reach, not multimode modal dispersion. One exact application is 800GBASE-DR8: eight parallel single-mode lanes over 16 fibers using an MPO-16 interface. A future 1.6T or 3.2T application can reuse an installed OS2 channel only when it passes this migration check: its lane and wavelength map, duplex or parallel fiber count, connector and polarity scheme, channel loss and reflectance, reach class, module and host requirements all work on the certified plant. OS2 headroom does not guarantee a transceiver-only upgrade. Choose the long-lived fiber and topology against an explicit migration map, then certify the installed channel and keep its records.

At 200G per lane the fork narrows. The IEEE P802.3dj draft defines its 200G-per-lane and 1.6T optical PMDs for single-mode fiber only. Multimode at that lane rate is a separate project, P802.3ds (PAR approved December 2025, submission for approval projected for August 2027), with reach objectives of at least 30 m and at least 50 m, up to 1.6T over eight fiber pairs. Its task force expects standard OM4 to carry the 30 m reach, while 50 m needs a new, higher-bandwidth multimode grade that IEEE asked IEC to define in March 2026. Installed OM4 can therefore carry 1.6T only on links of about 30 m or less, and only once that standard and its optics ship. A new multimode pull longer than that is a bet on a fiber grade that does not exist yet, so for a 1.6T route longer than about 30 m, pull OS2.

The reach classes map onto the fabric hierarchy. Inside the rack, copper still wins where it can: passive DAC on short tested routes, and active copper where its conditioning buys the extra span (Chapter 8.9). The worst-case NVL72 in-rack span is short enough to keep copper viable for scale-up today. Beyond that, single-mode DR over OS2 is an intra-hall candidate, FR/LR a longer campus candidate, and coherent ZR/ZR+ an inter-campus candidate, provided each application fits the installed distance, fiber, patching and repair budget. The inter-campus service belongs to scale-across (Chapter 8.8). The fiber plant is therefore layered: copper at the bottom for the cheapest, shortest, highest-volume links, and single-mode glass where the photon’s route or the migration plan outgrows what the tested multimode application can carry, with density and service access priced alongside reach.

A parallel-optic SR8/DR8 application carries eight optical lanes on separate transmit and receive paths, often arranged as a ribbon. A WDM FR4/LR4 application combines four wavelengths onto a duplex fiber pair. Keep the 800G or 1.6T rate and the exact PMD definition, including draft status where applicable, attached to that lane map. An 800G-DR8 link runs 8 transmit and 8 receive lanes, 16 fibers, terminated in a multi-fiber push-on connector (MPO/MTP). At 100k-GPU scale the fabric is a sea of these multi-fiber connectors. The way they are grouped into trunks, the polarity scheme that keeps transmit aligned to receive, and the connector count in each channel become the load-bearing engineering of the whole physical layer.

The base-N question is which fiber granularity the trunk is built around. Base-8 (MPO-8/MPO-12 wired as 8) groups fibers in eights. It divides evenly into four-pair applications and can feed a supported 400G/800G breakout without a conversion cassette when equipment cords and the actual port map match; count any stranded fibers and cassettes before claiming none. It is not a one-connector fit for an SR8/DR8 interface, which needs eight transmit and eight receive fibers, sixteen in all. A DR8 port is therefore served by two Base-8 groups whose pairing and polarity you draw explicitly on the connectivity map. Base-12, the legacy default, strands fibers or needs conversion modules when feeding 8-lane optics. MPO-16 carries 16 fibers in one connector, so an 800G-SR8/DR8 link fits end to end in a single ferrule. At a nominal 200G per optical lane, eight lanes give a 1.6T DR8 candidate on the same sixteen active fibers, but reusing the MPO-16 plant still requires the endpoint, connector and channel to pass that exact application. A later 3.2T step needs its own lane map and budget. Getting base-N wrong costs stranded fiber, conversion cassettes you cannot avoid (each adds a mated pair of loss), and a trunk whose breakout does not match the transceiver. Fixing that is days of rework across a hall.

Some 1.6T switch faceplates already use very-small-form-factor (VSFF) 16-fiber connectors such as MMC, whose ferrule takes under a quarter of an MPO’s footprint. Micas’s 102.4T Davisson CPO switch, an early preview at OCP 2026, exposes 64 × 1.6T MMC DR8 ports. Check which connector the equipment side uses before you order trunks. An MPO-16 trunk feeding an MMC faceplate needs a hybrid cord or cassette, and every added mated interface comes out of the same DR8 loss reserve. Inspection and cleaning tools and keyed end-face drawings must also cover both connector families.

Polarity, which ensures every transmit fiber lands on the far-end receive, is the silent source of acceptance failures. The TIA-568.3 methods are A, B and C for array connectivity, plus the universal U1 and U2 methods that revision E added for MPO-to-duplex breakouts. Each requires a disciplined, documented choice of trunk, cassette or adapter, and patch-cord type. Mix two methods in one channel and links go dark in ways that are tedious to isolate at scale. That is the strongest argument for structured cabling over point-to-point: a structured plant fixes one polarity method, documents it and uses keyed components, so a 100k-link fabric is repeatable and auditable rather than a per-link debugging exercise.

Lane-to-fiber migration record. Keep host electrical lanes, optical lanes per direction and installed fiber strands in separate fields. Record any gearbox, optical wavelength map and breakout grouping before you draw connector positions. A coded electrical signaling rate is not the nominal MAC service allocation, and neither number counts physical fibers. Attach keyed end-face drawings with pin gender and polish for each mating pair, and never derive the physical pin assignment from a generic Method A/B/C name.

The composed map carries all eight Tx/Rx pairs, so the assumed Base-16 trunk passes the fiber-count and polarity design checks. Its reserved-loss requirement is about 2.6 dB from Chapter 8.9. The two-group Base-8 alternative carries the same 16 strands, but the assumed conversions change four mated pairs to six. Two × 0.50 dB adds 1.0 dB, making the unrounded reserved requirement 3.640 dB, about 3.6 dB, which exceeds the current DR8 application ceiling. Keep the Base-16 route, or use equipment cords or conversion assemblies that restore the budget. Base-8 is not inherently ruled out: a four-pair arrangement with the same verified continuity and loss would reverse that rejection. The proposed 1.6T service is not approved until its own PMD limits and loaded tests pass.

The current pluggable in this assumed single-link record is one module FRU carrying one DR8 link: eight lanes in each direction on 16 fibers. Keep its cage and extraction clearance reachable without moving adjacent live fibers, and preserve a disjoint route during replacement. If migration moves the engine inside a switch, add the laser → engine → port → board dependency map and the actual FRU procedure before you accept reuse. A shared laser feeding both survivor paths would fail the repair requirement even when all 16 fibers pass. Obtain the stocked component and demonstrate drain, replacement and requalification within the owner’s outage allowance; neither an accessible ELSFP nor a compatible ferrule proves this. The DR8 application sets the fiber boundary and the ELSFP agreement the laser interface.

Structured or point-to-point is the last cabling decision, and it turns on install velocity, fault isolation and how the next density step gets executed. Point-to-point runs a jumper directly from one transceiver faceplate to another. It has fewer connectors (lower loss), but every move, add or change is a re-pull, fault isolation means tracing a specific jumper through a congested tray, and there is no patch field to re-patch a density change. Structured cabling lands trunks on patch panels (MDA/HDA cross-connects), with short equipment cords from panel to switch. It costs more connectors (more of the loss budget) in exchange for a documented, repeatable, auditable plant where a density step is a re-patch, a fault is isolated at a panel, and the install can be pre-terminated and pre-tested as factory trunks.

At AI scale the structured choice usually wins, for a reason specific to the density ramp: the trunk infrastructure is a long-lived candidate for reuse, while equipment cords, modules and host interfaces change faster. Reuse still holds only where the next application passes the fiber-plant section’s migration check on the installed plant. Structured cabling decouples the irreversible substrate (the trunk plant) from the reversible fit-out (the optics), the same reversible-versus-irreversible discipline that governs the slab and the power chain (Chapter 1.1). You pay the loss-budget and connector-count price up front to buy a plant you can re-patch instead of re-pull. The countervailing cost is that every mated pair eats the tight 800G/1.6T budget, so the loss-budget discipline above is non-negotiable and hyperscale AI plants pre-terminate, factory-test, and inspect and clean every connection as an acceptance gate.

Choose a structured channel only when its certified lane and polarity map, retained loss reserve and accessible FRU satisfy both this application and the planned migration. A simpler jumper or a different package wins when extra patching or shared optical dependencies consume those margins; paying for nominal future-proofing then buys stranded fibers and a harder repair.