The Archetype Decision Framework: Workload Is the Master Variable

Every other decision in this guide — the voltage you step down to, the temperature of the water you push to the rack, the oversubscription ratio on the back-end fabric, the redundancy tier you commission to, the county you site in — is downstream of one question that is almost never asked first: what is this machine for? Not "AI" in the abstract. The specific workload, its coupling, its tolerance for interruption, and its proximity requirement to the user. Answer that well and the rest of the lifecycle is a series of well-posed engineering problems. Answer it badly and you have spent two to four years and a capital stack on a facility that is mis-matched to the revenue it was supposed to earn — and most of those mistakes cannot be undone without re-pouring concrete.

This chapter is the framework that forces the question to the front. We define the five workload archetypes and the four procurement archetypes, then trace the requirements cascade: the deterministic chain by which an archetype propagates into density, cooling, fabric, storage, redundancy, siting, and TCO. We close on which decisions are reversible (defer them, keep optionality cheap) versus irreversible (over-build or hedge them now), the artifacts that capture a defensible scope, and the anti-patterns that recur because someone skipped this step.

Scoping an AI data center is two questions, and they are orthogonal — you must answer both, and the answer to one does not determine the answer to the other. WHAT will run here: pre-training, post-training/RL, online inference, batch inference, or edge inference (in practice, a weighted mix, but with a dominant archetype that sets the design basis). HOW will the capacity be procured and built: greenfield self-build, retrofit of an existing hall, wholesale or retail colocation, or rental from a GPU neocloud. A frontier pre-training run can be served from a self-build, a colo, or a neocloud; an online-inference business can equally be any of the four. The cross-product is a 5x4 matrix of real, deployed facilities — and the cost of mismatching a cell to its workload is the recurring theme of Part 1.

The reason this matters more in 2026 than it did in 2020 is that the binding constraint moved. The industry was chip-bound — the question was how many accelerators you could buy. It is now power-bound: the question is how many megawatts you can energize, and when. The US generator interconnection queue held more than 2,060 GW of active generation and storage capacity at the end of 2025 — over one and a half times the entire US installed fleet — with a median request-to-COD duration around five years for the generation/storage projects built in 2025 (LBNL). That supply-side population is not a data-center load-service queue; the project's energization date comes from its host utility's tariff, studies, upgrades, agreements and milestones. When power is the scarce input, a mis-scoped archetype does not just waste capital — it burns an interconnection slot you cannot get back, against a depreciation clock that is already running.

The five archetypes are distinguished by three properties that drive everything else: coupling (how tightly the accelerators must communicate within a single unit of work), interruption tolerance (whether the job survives a node failure cheaply or restarts from a checkpoint), and latency sensitivity (whether a user is waiting on the output in real time). Those three properties are the levers that translate "what runs here" into "how it must be built."

Pre-training is one tightly-coupled supercomputer. Thousands of GPUs run synchronous data-, tensor-, pipeline-, and expert-parallelism, dominated by all-reduce/all-gather collectives across the back-end fabric on every step. The whole job moves at the speed of its slowest straggler; a single failed GPU in a synchronous run forces a restart from the last checkpoint. This is the archetype that demands maximum density, direct-to-chip liquid cooling, a 1:1 non-blocking fabric, and the largest scale-up domains money can buy. → Chapter 1.2.

Post-training / SFT / RLHF / RL is the hybrid middle, and it is the most misunderstood of the five. Supervised fine-tuning is a small, bursty training job. But large-scale RL for reasoning is inference-heavy training: the dominant cost is generating rollouts — trajectories of 10K–100K+ tokens sampled from the current policy — not the gradient update that follows. That makes an RL cluster look like a fleet of inference engines feeding a comparatively small trainer, with asynchronous, staleness-tolerant coupling between the two. Scoping it as if it were pre-training over-provisions the fabric; scoping it as pure inference starves the policy update. → Chapter 1.4.

Online (interactive) inference is the revenue workload for most operators. It is bursty, latency-bound, and always-on: traffic can swing from 30% to 90% of capacity in minutes, and an SLO measured in time-to-first-token and time-per-output-token governs the user experience. It is loosely coupled (most requests fit inside a single node or a small scale-up domain), so the back-end fabric can be oversubscribed; but it demands high facility availability and proximity to users. Modern reasoning models, which emit long decode sequences, have inflated the decode share and the KV-cache pressure, reshaping fleet sizing. → Chapter 1.3.

Batch (offline) inference — embeddings generation, document processing, synthetic-data creation, evaluation sweeps — is throughput-bound rather than latency-bound. There is no user waiting, so it tolerates interruption, queuing, and aggressive oversubscription, and it is the natural consumer of spot capacity, off-peak power, and curtailable interconnections. It is the cheapest archetype to host and the most flexible to schedule.

Edge inference pushes serving to the user: on-prem appliances, telco/MEC nodes, Tier-2 metro colos, CDN-adjacent sites. It is defined by a hard latency budget (the 30/50/100 ms perceptibility thresholds) and severe power/thermal/space constraints, operated lights-out with zero-touch provisioning. Its siting driver is the inverse of pre-training: not cheap power, but physical proximity. → Chapter 1.5.

The table is a cascade. The leftmost two columns, archetype and coupling, are the inputs you control; everything to the right is a consequence. Choose pre-training and you have, in effect, also chosen liquid cooling, a non-blocking fabric, reinforced floors for ~3,000–5,000 lb wet racks (~3,000 lb for a single NVL72; higher on a multi-rack or forward-generation floor-loading basis), and a power-first siting search. Choose online inference and you have chosen a fundamentally different building: a latency-first site that may cost you 2–4x on energy, with fabric and KV-cache tiers sized to tail latency rather than step time. If maintenance or one component fault must not interrupt service, that outcome also selects a fault-tolerant power path; the workload name alone does not. The columns do not move independently — the archetype collapses a dozen subsystem decisions into one.

The second orthogonal question — how you acquire the capacity — trades four levers against each other: time-to-power, capital intensity, control, and workload duration. The fastest options surrender the most control and carry the highest unit cost; the cheapest-per-GPU-hour options demand the most capital and the longest lead time. The right answer is a function of how long you expect the workload to run and how certain you are about it.

Greenfield self-build gives maximal control over density, power architecture, and cooling — and the lowest long-run unit cost at scale — at the price of the longest, queue-gated schedule to a live cluster and the deepest capital commitment. It is the right call only for a durable, well-forecast workload at scale. Retrofit of an existing air-cooled hall trades capital and schedule for a hard physics ceiling: floor loading, plenum, electrical headroom, and available water cap how far you can push density. Colocation (wholesale or retail) buys time-to-power — a live 50k+ GPU cluster in a wholesale hall on the provider's schedule, not the grid's — by renting someone else's powered shell. Neocloud / GPU rental compresses time-to-first-job to however fast a provider can hand you capacity, and converts capex into opex, at the highest per-GPU-hour rate and the least control over the underlying fabric and reliability posture. The decision is rarely all-or-nothing: hybrids (burst-to-neocloud, colo-anchor-plus-cloud-overflow, build-core-rent-edge) are the norm. → Chapter 1.6; quantitative NPV in Chapter 1.8.

The cascade is a causal chain you can walk forward from a single input. It runs density → cooling → fabric → storage → redundancy → siting → TCO, and each link constrains the next.

Density sets cooling. This is the hardest constraint in the chain because it is physics, not policy. The cited GB200 NVL72 record assigns roughly 115 kW to liquid and 17 kW to residual air. That product-specific heat split and its airflow/inlet limits—not a universal rack-kW cliff—make direct-to-chip liquid necessary for this rack. The GB200 DLC envelope is tight but warm-water-capable: the rack accepts supply up to ~45 °C and return up to ~65 °C, while the separate ~170–235 L/min guide flow band is a water-property heat balance for a 7–10 °C design rise across the ~115 kW liquid share; Clariant PG25 at the same rise gives ~172–246 L/min, roughly 4% more. The 45/65 °C bounds are acceptance maxima, not the flow-sizing rise. Warm water is the design intent, and colder supply buys thermal headroom at the cost of chiller capex. Choosing the density target therefore commits the entire cooling plant, the facility water loop, and the heat-rejection strategy. → Chapter 5.1 (the density wall) and Chapter 5.4 (DLC).

Measured coupling sets the fabric. Published designs span 1:1, 2:1–3:1, and reported 7:1 examples; those are observations, not training/inference defaults. Derive the ratio per tier from the measured traffic matrix, collective/request mix, placement and communication overlap, topology, failure headroom, and step-time or tail-latency SLO, then validate it on the target fabric. Synchronous collectives often justify high bisection; local inference often permits upper-tier oversubscription, while distributed MoE, KV movement, and prefill/decode disaggregation can require more. The scale-up domain size (8 GPUs in HGX, 72 in NVL72, heading to 576) likewise follows the parallelism and failure-domain plan. → Chapter 8.5 (topology & oversubscription).

Interruption tolerance sets redundancy. For a synchronous training job, checkpoint-and-resume makes a single-path or component-redundant posture viable when restart loss stays inside the service objective; spend on stronger topology only when maintenance or fault events must not drop load. For an always-on inference business, lost-revenue and SLA exposure often require no load loss during maintenance or a single component fault; that outcome, rather than the workload name, selects the topology. This is the cleanest example of an anti-pattern: over-provisioned redundancy for checkpointable jobs buys nines the workload does not value. → Chapter 12.2 reframes this as goodput vs availability.

Latency sensitivity sets siting. Pre-training is indifferent to user proximity, so it chases the cheapest firm megawatts and the coldest free-cooling climate, accepting that the site may be hours from any metro. Online and edge inference invert this: they chase sub-50 ms reach to users and accept power that costs 2–4x more. Siting is the least reversible decision of all — you cannot move a slab — which is why it must be derived from the workload, never the other way around. → Chapter 3.1 (the reordered siting hierarchy) and Chapter 3.2 (speed-to-power).

Not all forks cost the same to re-decide. Sort decisions by the cost of changing your mind, and spend your optionality budget accordingly: over-build or hedge the irreversible decisions now; defer the reversible ones and keep them cheap to change.

Irreversible (decide once, at scoping): the site itself (you cannot move a slab); the grid interconnection capacity and voltage class (the queue slot is the scarcest asset in the project); the structural floor-loading basis (retrofitting a slab for ~3,000–5,000 lb wet racks mid-life is brutal); the base power architecture (415/480 VAC vs an 800 VDC path); and the macro cooling decision (a hall plumbed for liquid vs one that is not). These are the decisions where you pay to keep options open — e.g. provisioning floor loading and water for a density step-up you have not committed to yet.

Reversible (defer, re-decide cheaply): the specific accelerator generation within a power/cooling envelope; the scheduler and orchestration platform; oversubscription ratio on a fabric you sized non-blocking; the workload mix ratio within an archetype; and — critically — the procurement mode, which is why hybrid and rental exist. The strategic move is to convert irreversible decisions into reversible ones wherever the option premium is cheap: a powered shell instead of a full build-to-suit preserves IT-fit-out optionality; a colo lease preserves the option to exit; reserving floor loading and water headroom preserves a density ramp. → procurement framing in Chapter 1.6; refresh execution in Chapter 14.9.

A scope that survives board scrutiny and lender diligence is three artifacts that pin down the archetype decision and its consequences, signed before long-lead equipment is ordered.

• Workload profile sheet. The dominant archetype and the mix ratio; coupling, interruption tolerance, and latency budget; the implied scale-up domain size, GPU:CPU and GPU:storage ratios, and the back-end fabric blocking requirement. This is the single page from which the cascade is derived.
• Capacity ramp curve. MW and GPU count over time, generation by generation, with the density step-ups called out — because the ramp, not the steady state, is what the irreversible substrate (floor, power, water, cooling plant) must accommodate.
• Design-basis document. The frozen assumptions that everything downstream inherits: density tier, cooling modality, redundancy topology, voltage architecture, siting class, and the reversible-vs-irreversible register that records which assumptions are hedged and which are committed.

Per-archetype reference design-basis sheets and scalable-unit budgets are built out in Chapter 1.7; the economics that score the resulting scope live in Chapter 1.8.

The same mis-scopes recur, because each one comes from skipping the archetype question and reasoning from the equipment or the real estate instead. Three are worth naming explicitly:

• Sizing fabric from the workload label. A named model estimates a ~31% back-end cost difference between 1:1 and 2:1, but neither ratio is a training or inference default. Some node-local serving profiles may validate upper-tier oversubscription; distributed MoE, KV movement, or prefill/decode disaggregation may not. Derive each tier from measured traffic, placement, failure headroom, and the governing step-time or tail-latency SLO, then spend only validated savings on geo-distribution and uptime.
• Forcing a named rack beyond the inherited cooling envelope. Trying to land 132 kW liquid-cooled racks in a hall scoped for 40 kW air. The slab, the plenum, the power chain, and the absent facility water all say no. The retrofit either fails or strands capacity at a cost that would have funded a purpose-built liquid hall.
• Topology chosen from checkpointability alone. Checkpoint recovery changes the cost of an interruption but does not prove which maintenance or fault states may drop the hall. Define transfer interruption, post-event loading, path and control independence, common modes, recovery SLO, and contract before choosing N, N+1, distributed, or duplicated power. A topology is over-provided only when that state-based comparison shows a cheaper resilience path meets the same objective. → Chapter 12.2.