Utility Interconnect, On-Site Substation & MV Distribution

Chapter 3.2 brought you to the point of interconnection with a queue slot, a voltage class and a contracted megawatt figure, and Chapter 4.1 set the voltage and stage-count basis. This chapter follows those megawatts inside the fence, from the high-voltage yard through the on-site step-down transformers and the medium-voltage (MV) switchgear lineup, then across the campus to the pod and block transformers that feed each hall. It is the most physical and code-bound layer of the power chain, and the one whose consequences are easiest to miss. The white space gets the attention. The substation gets the schedule risk, the single points of failure, and the protection study that the responsible engineer must deliver and that a lender reviews where the financing contract requires it.

The consequences here are unforgiving because they are geometric and regulatory at once. A radial bus saves you switchgear sections and tie equipment, and costs you the whole downstream hall on a single breaker failure. An SF6 gas-insulated lineup can shrink switchgear to as little as about 30% of the air-insulated footprint (ABB's figure for its 38 kV product), but in the EU, from 1 January 2026, putting such equipment into operation is prohibited at ≤24 kV, subject to Regulation (EU) 2024/573 Article 13 exceptions. A relay scheme tuned for stiff utility fault current trips correctly on the grid and may refuse to trip when you island on inverter-based sources such as batteries (BESS), solar and fuel cells. Grid-following inverters, for example, may supply only about 1.2–1.5 times rated current; that range is illustrative, not a universal source model. You walk through each of these forks once, at design, and live with it for the 20-to-30-year life of the yard.

The on-site electrical estate of a large AI campus is a two-stage step-down with a distribution fabric in the middle. HV transmission (138/230/345/500 kV in ANSI markets; 132/220/400 kV in much of the IEC world) lands in the customer substation. There HV-to-MV power transformers step it down to a medium-voltage bus. They are typically tendered as multiple HV/MV units, sized from the facility MW, power factor, site rating and surviving-unit count. The 150 MW, three-unit capacity case in Chapter 4.4 shows why a nominal 80–100 MVA range is not a specification. From that MV bus (commonly 34.5 kV, 33 kV, 22 kV, or 13.8/11 kV depending on region and campus size), MV distribution carries power across the site to pod/block transformers. These step down to the LV utilization voltage the racks consume (400/230 V IEC or 480/277 V ANSI). An emerging alternative steps down straight to an 800 VDC rail with a solid-state transformer; as of 13 October 2026 that path is at prototype, pilot and early commercial stage (see Chapter 4.4 and Chapter 4.7).

Campus size sets the service voltage. Distribution-class service tops out near 20–40 MW, 69–138 kV sub-transmission carries a 100–300 MW campus, and 230/345/500 kV transmission is the entry ticket for the gigawatt factories. This chapter owns everything between that incoming voltage and the pod transformer: the switchgear that makes and breaks the MV bus, the topology that connects sources to loads, the relays that protect it, and the coordination study that proves it is safe and selective. HV-yard ownership, NERC registration and utility-boundary operations are in Chapter 4.3. Grid-interactive ride-through and reactive support are in Chapter 4.10.

Size the site's own step-down the way the utility rates its substation: firm capacity is what remains with the largest transformer out of service. Two 100 MVA units give 100 MVA firm, not 200. A load above 100 MVA that must stay energized through a transformer failure or maintenance outage therefore needs a third unit or a tie to another bank, and that choice sets the HV bay count, the yard and the long-lead order. Chapter 3.3 applies the same N−1 rule to the serving substation and screens line loadability by voltage class.

The first real equipment fork is the switchgear technology that builds the MV bus: air-insulated (AIS) or gas-insulated (GIS). AIS uses air as the dielectric between live parts, so it costs less per bay, is easy to inspect and can be serviced by any competent MV crew. The price is footprint, because air needs clearance. GIS encapsulates the busbars and switches in grounded metal enclosures filled with an insulating gas. That seals them against dust, salt and humidity and removes weather exposure, and it shrinks the footprint: ABB advertises up to a 70% reduction against AIS for its 38 kV ZX switchgear, about 30% of the AIS area. GIS also costs more and takes longer to deliver. Its gas-service burden depends on the assembly. ABB says its sealed ZX panels need no gas work on site at any stage, so write down the gas-handling and service needs of each assembly you buy. At HV, expect gas handling and specialist crews.

For a land-rich greenfield in a temperate climate, AIS is often the rational default: cheaper, faster to source and easier to operate. GIS justifies its premium where footprint is the binding constraint (urban or constrained sites, where every square meter of yard is a square meter not spent on white space), where the environment is hostile (coastal salt fog, desert dust, the Gulf and Singapore buildouts), or where a compact, weather-sealed yard shortens the civil works on the critical path. The decision is not purely electrical. It trades footprint against capital and schedule, and it interacts with the site constraints from Chapter 3.1.

What changed the calculus is the insulating gas. Conventional GIS used sulphur hexafluoride (SF6), a superb dielectric and arc-quenching medium and the most potent greenhouse gas regulated under the Kyoto basket, with a 100-year global-warming potential of about 24,300 times that of CO2 (IPCC AR6, also the value in the adopted EU regulation). The EU F-Gas Regulation (Regulation (EU) 2024/573, Article 13(9)) now bans putting into operation new switchgear that uses or relies on fluorinated gases in its insulating or breaking medium: up to 24 kV from 1 January 2026, >24–52 kV from 2030, and, for any F-gas with GWP ≥1, >52–145 kV (≤50 kA) from 2028 and >145 kV (or >50 kA) from 2032. So the EU requirement is gear free of all F-gases, not just SF6: it applies now to MV equipment at 24 kV or less, and from 2030 to the >24–52 kV band, which includes 33 kV. A 33 kV lineup put into operation before 2030 can still use SF6, but it commits the yard to a gas the regulation is phasing out, so check spares, gas-service and extension terms before you buy. Because the major OEMs build to one global platform, SF6-free is also becoming the default offering worldwide. The same regulation restricts high-GWP refrigerants in new chillers from 2027 (Chapter 5.8).

Two details change how that gate applies. First, air insulation alone does not settle compliance: the rule covers the breaking medium too, and some air-insulated panels still use SF6 breaking technology (Schneider Electric's SM6-24 DM1A is one). Second, the exceptions follow the procurement outcome. Run a gas-neutral tender. Article 13(11) lets you buy equipment with GWP below 1,000 when the tender draws no compliant bid, or when only one compliant manufacturer bids within two years of the prohibition date. If even that is unobtainable, Art. 13(12) permits SF6. Orders placed before 11 March 2024 and incompatible extensions have separate exceptions. You must notify the authority when you use Arts. 13(11), (12), (14) or (15), and keep the derogation records for five years. Operation generally starts at handover after tests and inspections, although the Commission's FAQ allows a completion-based date for qualifying large projects.

California has its own binding rule. Under the California Air Resources Board's rule (17 CCR §95352, as of 13 October 2026), buying new SF6 gas-insulated equipment is being phased out, with listed exemptions and grandfathering. The date is 1 January 2025 for aboveground gear rated below 38 kV and for gear above 38 kV through 145 kV rated below 63 kA. Aboveground 38 kV gear and the above-38 kV through 145 kV band at 63 kA or more follow in 2028. Higher-voltage and belowground equipment follow their own dates in Tables 1 and 2 of the rule. Fluoronitrile mixes with GWP above 1 fall under the rule's emissions and reporting provisions, not the SF6 acquisition phase-out. Check an SF6 specification's exceptions and acquisition history before you call it unlawful.

This topology decision sets the blast radius. Three families dominate, and they trade capital and complexity directly against fault tolerance and concurrent maintainability.

Radial is the simplest and cheapest: one source feeds a bus feeds a load, with no alternate path. A fault or maintenance outage anywhere upstream drops everything downstream. It is right only where the load tolerates the outage: batch-inference halls, non-critical mechanical, or any block where a trip is an inconvenience rather than a revenue event. Build a revenue inference hall on a pure radial and every MV maintenance window is a planned outage you cannot avoid.

Primary-selective and secondary-selective give each load two sources, a normal feed and an alternate, with a manual or automatic transfer between them. Secondary-selective is the workhorse of the critical data center: dual MV feeds into a double-ended substation with a normally open tie breaker. When a source feeder or transformer fails, isolating it lets the healthy side carry the retained bus. A transfer brings a healthy bus back after an interruption. It does not keep supply uninterrupted. A fault on the load's own bus or on a downstream pod transformer is a different zone, and that load stays out until an independent path or a repair exists. This topology delivers concurrent maintainability: you can de-energize one side for service while the other carries the hall. It is the natural MV partner to a 2N or block-redundant power chain (see Chapter 12.1). Its cost is the second feeder, the second transformer, and the tie switchgear and transfer scheme.

Ring (and ring-main / breaker-and-a-half at the higher end) connects the distribution into a loop so that any segment can be isolated while power still reaches every other load from the opposite direction. What a ring buys depends on how it is operated and protected. A closed ring with directional and differential protection can clear a faulted segment and keep unfaulted loads fed, on less total feeder copper than a fully duplicated radial pair. The cost is the protection scheme: with sources at multiple points, the relays must work out which way fault current is flowing, and the load still sees whatever disturbance the clearing causes. An open (normally open) ring is a restoration path, not an uninterrupted one: loads on the faulted section stay out until the fault is isolated and supply is brought round from the other direction. A ring cannot feed a load bus or downstream transformer that is itself faulted. Specify the protection zones, the sectionalizing and restoration scheme, and the disturbance the load may see before you call a ring fault-tolerant.

For gigawatt campuses, the HV yard itself often goes to a breaker-and-a-half or double-bus arrangement. Breaker-and-a-half lets you maintain any one breaker without an outage, but a breaker failure still costs circuits while breaker-failure protection clears it. An outer-breaker failure drops one circuit. A middle-breaker failure drops both circuits in that diameter until they are isolated and restored. Land each redundant campus supply in a different diameter, then show that the paths left after clearing can carry the load. "Double bus" alone guarantees neither breaker redundancy nor campus continuity, so write down the real breaker arrangement and its clearing zones.

Read this table against the workload archetypes from Chapter 1.1, not against a generic Tier number. If checkpoint recovery keeps restart loss within your uptime target and you can accept MV maintenance or fault interruptions, the ring premium may return less than checkpoint cadence and hot spares would (the goodput-vs-availability argument in Chapter 12.2). If every MV trip breaches the SLO and loses revenue, require a secondary-selective or ring topology that limits the interruption. Even then the downstream UPS has to bridge it: a secondary-selective transfer or open-ring restoration leaves a gap, a closed ring leaves a fault-clearing disturbance, and transfer into a faulted retained bus must be blocked. The common mis-scopes are a pure radial beneath a no-load-loss revenue objective, and a fully duplicated, ring-fed MV fabric beneath a restart-tolerant objective that accepts an MV interruption.

Switchgear and protection come in two standards lineages. Region sets which one you build to, but it is still a real design fork, because it reshapes the relay panel, the ratings, the test regime and the spares. The IEC lineage (IEC 62271 for switchgear, IEC 60255 for relays, IEC 61850 for substation communications) governs most of the world outside North America. The ANSI/IEEE lineage (IEEE C37 for switchgear and breakers, ANSI device numbers, NEC/NFPA 70 for installation) governs the US and ANSI-aligned markets. Both are independent of grid frequency. They differ in standard voltage steps, fault-duty definitions, rating philosophy, and the device numbering and documentation your relay engineers and operators live in. A global operator that builds one campus to IEC and the next to ANSI runs two parallel protection practices, with different relays, test sets, spares and qualified crews.

Cutting across both is IEC 61850, the substation-automation standard that reshaped how protection is wired. Instead of a relay panel hardwired with copper from every CT, VT, breaker and trip coil, 61850 puts the traffic on a redundant Ethernet network. The station bus carries MMS for SCADA and engineering, plus GOOSE for fast inter-relay tripping and interlocking. The process bus adds Sampled Values, which are digitized CT and VT measurements, and can carry GOOSE for process-level binary signals too. A station-bus-only design can therefore use Ethernet GOOSE while the CT and VT analog signals stay hardwired. The payoff is less copper, faster and testable interlocking, vendor-interoperable relays described by one SCL configuration file, and a substation you can model and FAT digitally. The cost is that protection now depends on a deterministic network: GOOSE messages must arrive in a few milliseconds and the network needs PRP/HSR redundancy. A 61850 substation is also a networked OT asset that belongs in the site's security architecture (Chapter 11.10). NERC CIP applicability follows the owner or operator's registered function, the facilities CIP-002 lists, and how the BES Cyber System is categorized (Chapter 4.3). The protocol does not decide it, and neither does who owns the substation. CIP-002-5.1a is the edition in force on 13 October 2026. The 2026 default for a new large substation is a 61850 station-bus design; full process-bus adoption is the frontier, traded against the maturity of your protection vendor and crew.

The campus is also a grid actor bound by the regional grid code, and that obligation reaches back into this layer through the protection settings. The grid code sets the voltage and frequency range your protection must not trip within, the inverse of normal protection's job and the source of real tension. In Europe the load connection falls under the Demand Connection Code, and any on-site generation that runs in parallel with the grid falls under the Requirements for Generators (RfG), both EU network codes drafted by ENTSO-E. Ireland goes further: its Grid Code modification MPID345, effective 27 October 2026, requires transmission-connected demand to ride through faults and restore at least 90% of pre-fault demand within 500 ms of voltage recovery (Chapter 3.2). The GB market adds the NESO Grid Code (the National Energy System Operator took over from National Grid ESO on 1 October 2024).

North America adds NERC reliability standards and regional rules, and Northern Virginia shows the failure they are meant to prevent. In July 2024 a six-fault 230 kV sequence sent sites to backup, and the grid lost about 1,500 MW of data-center load. On 22 July 2026 in Ashburn, one normally cleared fault sent sites to backup again, and about 3.8 GW left the grid (PJM). Before Ashburn, in May 2026, NERC had already issued a Level 3 "Essential Actions" alert. Its addressees are grid entities such as transmission planners, owners and operators, and it places no obligation on data centers.

As of 19 October 2026, ride-through is binding in ERCOT, and in SPP for large-load requests filed since 15 January 2026. ERCOT's NOGRR282 and NPRR1308 have, since 1 August 2026, required covered Large Computational Loads (at least 75 MW, with computational load making up half or more of demand) to ride through voltage and frequency disturbances. NERC announced on 15 October 2026 that its computational-load standards (CLO-001, CLO-002 and CLO-003) had passed final ballot. They set no ride-through curve and still need NERC Board adoption and FERC approval, and FERC's 16 July 2026 order in RD26-7-000 requires the filing by 31 December 2026. PJM proposed its own large-load ride-through rules on 8 September 2026 and plans to file them at FERC in November. Take the applicable regional curve from Chapter 4.10.

The pod (or block) transformer is the unit of capacity you energize, fault, protect and maintain. Its size is a granularity decision with direct consequences for blast radius and for how the density ramp lands. A few large transformers minimize part count, footprint and unit cost and simplify the MV bus, but each is a larger single point of failure, and each strands more capacity while a hall is half-populated during ramp. Many smaller transformers shrink the blast radius (a transformer fault drops a pod, not a hall), let you energize capacity in finer steps that track the GPU delivery curve, and fit a pod-as-fault-domain design (see Chapter 12.1). The cost is more bays, more relays, more footprint and a higher aggregate capex.

The MVA math is unglamorous and decisive. The 150 MW facility case in Chapter 4.4 treats the 10% planning margin, 0.95 power factor and 0.95 site factor as independent inputs. Under them, three 80 MVA units fail N−1 and three 100 MVA candidates pass its capacity screen. The minimum is about 91 MVA per unit under those assumptions, not a universal 80 or 82.5 MVA prescription. You still have to set the pod transformer's own IT/cooling boundary, load sharing and surviving bus path. A unit that fits the normal-state total can still strand the retained hall after a transformer outage, so keep that distinction when you carry MVA from the tender sheet onto the one-line.

The figure that should haunt the sizing exercise is the density ramp. A pod transformer sized for the 40–60 kW air-cooled racks that are still current product lacks the headroom for GB300 NVL72 at its OEM peak or busway basis (Lenovo rates up to 155 kW peak, HPE recommends 192 kW busway provisioning, and NVIDIA's reference architecture says up to 142 kW). It lacks it even more for Vera Rubin NVL72 at NVIDIA DSX's 330 kW cabinet facility design basis, or the ~600 kW Rubin Ultra / Kyber roadmap planning point. Transformer lead times run past 100 weeks, so you cannot swap one on a quarter's notice (see Chapter 2.3; Chapter 4.4 carries the transformer lead-time figures). The doctrine is the one that governs floor loading and water in Chapter 1.1: provision the irreversible MV substrate (feeder ampacity, transformer pad and MVA headroom, switchgear bus rating) for the ramp you can foresee, and keep the reversible LV fit-out matched to the generation actually shipping.

Protection decides, in milliseconds, that something has faulted, and it isolates the smallest possible piece of the network to clear it. On an MV campus it is built from numerical relays running a layered set of functions, referenced by ANSI device numbers in the US lineage and by IEC 60255 function names in the IEC one. They are coordinated so that the relay closest to the fault trips first and the ones upstream hold, which preserves as much of the network as possible. The core elements on a data-center MV system are overcurrent and earth-fault (ANSI 50/51, 50N/51N) on every feeder; differential protection (87T on transformers, 87B on busbars), which clears in-zone faults instantaneously without waiting for time grading; directional elements (67) wherever current can flow two ways (rings, paralleled sources, on-site generation); and transfer and bus protection tied into the automatic source-transfer scheme of a secondary-selective substation.

In a 61850 station, much of the interlocking and fast tripping that used to be hardwired now rides on GOOSE messages. A relay that sees a busbar fault publishes a trip that other relays act on in a few milliseconds, which enables fast bus-blocking and breaker-failure schemes without copper between panels. This is faster and testable, but it makes protection a function of the substation network, which is why the PRP/HSR redundancy and the security posture matter. Protection is what makes the topology's redundancy real, so engineer it with the single-line, not after it. A normally open ring is protected as its operating radial sections. A closed ring needs a scheme proven for its actual infeeds, zones and switching states. Select the differential and directional functions from that study, not from the word ring.

The core paper artifact of this layer is the protection and coordination study. It is a controlled engineering deliverable, reviewed by the responsible engineer and submitted wherever the authority, insurer, lender or contract requires it. It has four canonical parts, each a study in its own right.

• Short-circuit / fault-duty analysis. Computes the available fault current at every bus, from the utility's source impedance through the on-site transformers and generation, and proves every breaker and bus is rated to interrupt and withstand it. Under-rate a breaker against the available fault current and it can fail catastrophically trying to clear a fault, the most dangerous error in the building.
• Time-current-curve (TCC) coordination. Following the active IEEE 3004 dot-standard that covers your equipment and study scope, plots every protective device's time-current characteristic so that, for any fault, the nearest upstream device trips first and the ones above it hold. That is selectivity. Mis-coordinate and a feeder fault trips the main, turning a pod outage into a hall outage.
• Arc-flash incident-energy analysis. Per IEEE 1584 and NFPA 70E (and CSA Z462 / regional equivalents), computes incident energy and the arc-flash boundary at each working distance, so arc-rated PPE and equipment labels match the calculated exposure. NFPA 70E does not let you convert that result into a PPE category or mix the two methods on one piece of equipment, and shock approach boundaries are set separately by voltage. This is a life-safety deliverable, not a formality. It ties into the qualified-worker EHS interface of Chapter 6.9.
• Relay setting sheet. The pickup, time-dial and curve-type settings derived from the above, which the commissioning team loads into every relay and the operations team maintains for life.

The study is not one-and-done. Re-run it whenever the source impedance changes, such as a new utility line, a different transformer or on-site generation, because each of those moves the fault current and can break coordination that was selective yesterday. In the BTM era, the change that breaks it most often is the addition of inverter-based on-site power.

The open tie passes this one duty screen. Referred to 400 V, define Zs = 400²/(500 × 10⁶) Ω and Zt = (400²/630,000) × 0.06 Ω. Evaluate Iopen = 400/[√3 × (Zs + Zt)] and Itied = 400/[√3 × (Zs + Zt/2)] from those full expressions, which give about 15 kA open and 29 kA tied. Only the transformer branch impedance halves, because the upstream source is common. A 25 kA candidate therefore survives the first comparison and fails the tied state. Varying transformer impedance from 5% to 7% and the source from 250 to 750 MVA keeps the source much stiffer than either transformer, and only near the weak corner does the tied duty fall below 25 kA. With 7% transformers it crosses 25 kA once the source drops below about 460 MVA, and at 250 MVA it is about 24 kA. So the study must use the maximum fault level the utility quotes, not a typical value.

The crossover is a total positive-sequence impedance of 400/(√3 × 25,000) ≈ 0.0092 Ω; compare against the unrounded expression. If the total falls below it, the candidate fails its symmetrical duty screen. Keep the tie open and interlocked, or tender a higher-duty assembly and re-study every operating state.

At AI-hall scale the tie decision is forced. The 630 kVA unit keeps the arithmetic small, but a GPU pod's LV block is several times larger: Vertiv's 360AI reference design serves three NVIDIA DGX SuperPODs, 3.4 MW in all, with 4,000 A main switchboards and 2,500 kVA UPS units, three of each. Rerun the screen with a 3 MVA, 6% transformer and it gives about 66 kA with the tie open and 120 kA with it closed. The closed state then needs gear rated and re-studied above 120 kA, or an interlock that never lets both transformers feed one fault.

In the final study, the protection engineer replaces every assumption here with utility maximum and minimum fault data, voltage factors, transformer tolerances, hot and cold conductor impedances, and rotating and inverter contributions. Breaker peak making duty needs its own check, and so do selectivity, clearing time, short-time withstand, CT performance and minimum-fault pickup. The open-state current is neither an arc-flash result nor proof that a relay trips. Before energization, the electrical-operations handover (Chapter 4.12) needs the revised one-line, the approved settings, a tested tie interlock and witnessed trip records.

At campus scale the topology and granularity decisions compose into a handful of recurring single-line patterns, and the pattern scales with the megawatt count. A 100–300 MW campus typically takes a 69–138 kV interconnection into a customer substation with a double-ended (secondary-selective) MV bus. That bus feeds a small number of MV distribution rings or selective feeders out to block transformers, which gives enough redundancy for concurrent maintainability without the cost of a full transmission yard. A 500 MW–1 GW factory connects at 230/345/500 kV, and its HV yard is laid out as breaker-and-a-half or double-bus, with redundant supplies in separate diameters and the clearing zones written down, as described above. The factory also has multiple HV/MV transformer banks and an MV backbone, often a ring or a set of selective feeders, that subdivides the campus into independent pods or blocks. Each is its own fault domain with its own transformer, switchgear and protection zone.

The unifying design principle across the scale range is fault-domain alignment: the MV architecture should partition the campus so that one electrical fault stays inside one workload-meaningful unit (a pod, a hall, a block) and never propagates further. That partition should match the fault domains the resilience design wants (see Chapter 12.1), the granularity at which you energize capacity during the density ramp, and the workload's tolerance for losing that unit. When the MV blast radius, the energization increment and the workload fault domain are the same boundary, the substation is doing its job. When they diverge, because a fault drops more than the workload can absorb or you can only energize in increments bigger than your GPU delivery, the single-line is fighting the building.