The Definitive Guide toAI Data Centers

17 Parts · 173 chapters

0Foundations & How to Use This Guide

0.1Orientation: The AI Data Center as a Single Co-Designed Machine 0.2How to Read This Guide: Decisions, Consequences & Reference Data 0.3Vocabulary, Mental Models & the Metric Stack 0.4The Standards & Specifications Landscape (Living Index)0.5Reliability, Redundancy & Availability: The Design-Basis Primer

1Strategy, Workload Archetypes & Economics

1.1The Archetype Decision Framework: Workload Is the Master Variable 1.2Training Data Centers: Synchronous, Dense, Checkpointable 1.3Inference Data Centers: Bursty, Distributed, Always-On 1.4Post-Training, Fine-Tuning & RL: The Hybrid Middle 1.5Edge Inference & Distributed Micro-Datacenters 1.6Procurement Archetypes: Build vs Buy vs Rent 1.7The Requirements-and-Consequences Matrix 1.8Business Models, Economics & ROI

2Project Delivery, Schedule, Procurement, Contracts & Risk

2.1Program & Project Management: The Integrated Master Schedule & Critical Path 2.2Delivery Models & the Owner's Organization 2.3Long-Lead Procurement & the End-to-End Equipment Supply Chain 2.4The Contract Stack & Commercial/Legal Framework 2.5Project Finance & Capital Formation (Mechanics)2.6Insurance & Risk Transfer 2.7Simulation-Driven Design & the Digital Twin as a Design-Validation Tool

3Site Selection, Power Procurement & Permitting

3.1Site Selection Strategy & the Reordered Criteria Hierarchy 3.2Grid Interconnection, Queues & Speed-to-Power 3.3Power Availability & Power-Cost Structure 3.4Energy Supply Strategy: Grid PPA, BYOP & Co-Location 3.5On-Site & Bring-Your-Own-Power Generation (Energy-Supply Strategy)3.6Fiber, Latency & Network Connectivity (Secondary Screen)3.7Water Availability, Sourcing & Climate-Driven Cooling Strategy (Siting Gate)3.8Land, Geotechnical, Seismic & Flood Diligence (Secondary Screen)3.9Permitting, Regulatory, Environmental & the Critical Path 3.10Tax Incentives, Fiscal Structuring & Economic Development 3.11Community Relations, Opposition & Social License 3.12Geopolitics, Sovereignty, Export Controls & Data Residency 3.13Market Clusters & the Site-Scoring Playbook

4Electrical & Energy Infrastructure

4.1Power Topology Foundations & Voltage Selection 4.2Utility Interconnect, On-Site Substation & MV Distribution 4.3Substation & Transmission Ownership, Operations & NERC Compliance 4.4Transformers, Harmonics & the AI Non-Linear-Load Problem 4.5UPS & Energy Storage: From Ride-Through to Transient Absorption 4.6LV Distribution: Busway, PDUs, RPPs & Rack Power 4.7The DC Power Revolution: 48V → ±400V → 800V & Disaggregated Sidecar Power 4.8On-Site Generation: Electrical Integration 4.9Fuel-Supply & Gas-Process Engineering 4.10Grid-Interactive Behavior: Ride-Through, Reactive/Voltage Support & Frequency Response Toward the POI 4.11Grounding, Bonding, Earthing, Lightning Protection, SPD & EMC 4.12Metering, Power Quality, Monitoring & Electrical Operations

5Cooling & Thermal Management

5.1Thermal Fundamentals & the Density Wall 5.2Air Cooling at the Limit 5.3Rear-Door Heat Exchangers & Air-Assisted Liquid Cooling (The Bridge)5.4Direct-to-Chip Liquid Cooling (DLC) — The 2026 Default 5.5Immersion Cooling (Single-Phase & Two-Phase)5.6CDUs & the Secondary Loop 5.7Facility Water Loops & Warm-Water Cooling 5.8Heat Rejection: Chillers, Dry Coolers, Towers, Adiabatic & Economizers 5.9Heat Reuse & Waste-Heat Recovery (Engineering)5.10Retrofitting Air-Cooled Facilities for Liquid 5.11Thermal Design, Reliability, Leak Detection & Commissioning 5.12Cooling-Controls Transient Dynamics & Setpoint Stability 5.13Facility Piping & Pressure-System Mechanical Engineering

6The Building: Civil, Structural, Fire/Life-Safety & Construction Execution

6.1Building Typologies & Data-Hall Layout 6.2Structural & Civil Engineering for Dense Liquid-Cooled Halls 6.3Building Envelope, Architecture & Site Civil Works 6.4Modular & Prefabricated Construction 6.5Fire Detection, Suppression & Life-Safety 6.6Construction Execution, Sequencing & Phased Turnover 6.7Rack Civil Integration: Mass, Floor-Loading & Seismic Anchoring 6.8Acoustic & Emissions Engineering Design 6.9Environment, Health & Safety (EHS) Across Build & Operate

7Compute, Silicon & System Integration

7.1Accelerator Landscape & Taxonomy 7.2NVIDIA Accelerators: Hopper → Blackwell → Vera Rubin → Rubin Ultra → Feynman 7.3AMD Instinct & the Open Challenger 7.4Hyperscaler XPUs: TPU, Trainium/Inferentia, Maia, MTIA 7.5Custom ASICs & the Merchant-Silicon Disruption 7.6HBM: The Binding Constraint on AI Compute 7.7Advanced Packaging & the Integration Substrate 7.8Host CPUs, GPU:CPU Ratios & System Composition 7.9Software Ecosystems & Lock-In 7.10Precision, Quantization & the Compute-Memory Tradeoff 7.11Accelerator Selection, TCO & Procurement Strategy 7.12On-Package Power Delivery & Power Integrity 7.13The Rack as Integration Unit 7.14Server & System Integration 7.15Deployment Velocity & Cabling at Scale

8Networking, Fabrics & Optics

8.1Network Fundamentals & AI Traffic Characterization 8.2Scale-Up Fabric (Intra-Node / Intra-Rack)8.3Network Silicon: Switch ASICs, NICs & DPUs 8.4Scale-Out Fabric: Protocols, Standards & Transport 8.5Scale-Out Topology, Sizing & Oversubscription 8.6Congestion Control, Load Balancing & In-Network Compute 8.7Management, Out-of-Band Fabric & PTP/IEEE-1588 Timing 8.8Scale-Across: Multi-Campus & Cross-Region Fabric (DCI for Distributed Training)8.9Physical-Layer & Interconnect Taxonomy 8.10CPO, Fiber Plant & Structured Cabling

9Storage & Data

9.1Storage in the AI Lifecycle: Why It Determines GPU Efficiency 9.2Parallel & Distributed File Systems 9.3NVMe Tiers, GPUDirect Storage & the CPU-Bypass Data Path 9.4Checkpointing for Large-Scale Training 9.5Data Ingestion, Preprocessing & the Data-Loader Path 9.6Object Storage, Data Lakes & the Capacity Tier 9.7Inference & KV-Cache Storage: The New Memory Hierarchy 9.8Sizing, Data Gravity & Resilience 9.9The Data-Prep Supercomputer: Offline Data Processing

10Software, Orchestration & Service Delivery

10.1Orchestration Architecture & the Scheduling Plane 10.2Topology-Aware & Rack-Scale Scheduling 10.3Multi-Tenancy, Isolation & Resource Sharing 10.4Node Software Stack: Drivers, CUDA/ROCm, NCCL & Firmware 10.5Provisioning, Bring-Up & Infrastructure as Code 10.6Observability, Telemetry & GPU Health 10.7Fleet Reliability, Fault Tolerance & Autonomous Recovery 10.8MLOps & Training Frameworks 10.9Customer Onboarding, Delivery & Productization 10.10Data Governance, Privacy & the Training-Data Legal Regime 10.11Inference Serving Engineering: SLOs, Batching, Disaggregation & Goodput-Optimal Scheduling

11Security

11.1Threat Model, Assets & Security Levels for AI Infrastructure 11.2Physical Security: Siting, Zones & Kinetic/Drone Threats 11.3Supply-Chain Security & Hardware Provenance 11.4Hardware Root of Trust, Firmware & BMC Security 11.5GPU Confidential Computing & Trusted Execution 11.6Multi-Tenant & Workload Isolation Security 11.7Network Segmentation, Microsegmentation & Zero Trust 11.8Model & Weight Protection (At-Rest, In-Transit, In-Use)11.9Insider Threat & Human-Layer Security 11.10Cyber-Physical & Destructive Attacks on OT/Facility Systems 11.11Compliance, Certification & Governance 11.12Security Operations, Detection & Incident Response

12Reliability, Resilience & Standards

12.1Resilience Standards, Redundancy Topologies & Fault-Domain Engineering 12.2The AI-Cluster Reliability Rethink: Goodput vs Facility Availability 12.3Disaster Recovery, Business Continuity & Geographic Failover 12.4SLAs, Goodput Contracts & Availability Commitments 12.5Quantitative Reliability & Availability Modeling (RBD / FTA / Monte-Carlo)

13Commissioning & Go-Live

13.1Commissioning Fundamentals, Levels & Program Governance 13.2Documentation, Scripts & Acceptance Test Plans 13.3Electrical Power Acceptance (L3/L4)13.4Commissioning On-Site Generation & Microgrid Controls 13.5Cooling Acceptance: Air, Liquid-to-Chip & CDU Commissioning 13.6Level 5 Integrated Systems Testing (IST) & Failure-Mode Demonstration 13.7Network Fabric Commissioning & Validation 13.8GPU Node Burn-In, Diagnostics & Stress Validation 13.9Cluster-Scale Benchmarking, Reference Training & Storage/Scheduler Validation 13.10Staged Power/Load Ramp, Go-Live & Handover to Operations

14Day-2 Operations, Upgrades & Lifecycle

14.1Operational KPIs, Goodput & the Reliability Economics of AI Factories 14.2DCIM, Telemetry & Observability for GPU-Dense, Liquid-Cooled Facilities 14.3Component Failure Modes, Failure Rates & Fleet Reliability Data 14.4Reliability Engineering for Training (Operational)14.5Predictive & Preventive Maintenance of Power and Cooling Plant 14.6Spares Strategy, RMA Logistics & Repair Operations 14.7Capacity, Power & Thermal Management in Operation 14.8Firmware & Software Lifecycle Management at Fleet Scale 14.9Hardware Refresh, Depreciation Strategy, Decommissioning & ITAD 14.10Facility Decommissioning, Repowering & Site Remediation 14.11Operations Organization, Workforce, Talent & Incident Command 14.12Operational Procedures, Change Management & Human-Error Control 14.13Agentic Ops, RL Control & the Autonomy Ladder 14.14Continuous & Re-Commissioning on a Live Campus

15Sustainability & Efficiency

15.1Efficiency Metrics: PUE, WUE, ERF, REF & the Post-PUE Metric Stack 15.2Energy Efficiency: Cooling, Free Cooling, Setpoints & Power-Chain Losses 15.3Carbon, Clean Power Procurement & 24/7 Carbon-Free Energy 15.4Water Stewardship 15.5Heat Reuse & District Heating (Sustainability & Economics)15.6Embodied Carbon & Circularity Across the Lifecycle 15.7Regulation, Reporting & Disclosure Frameworks 15.8Grid Impact, Energy-Systems Integration & Grid Services

16Trends, Roadmaps & the Future

16.1The Power-Bound Era: Why the Bottleneck Moved to the Substation 16.2Subsystem Roadmaps 2026 → 2030 (Consolidated)16.3Software, Orchestration & Efficiency at the Frontier 16.4The Economics of the Build-Out 16.5Scenarios for 2030

§Appendices & Reference Data

AStandards & Specifications Cross-Reference Matrix BReference Designs & Worked Examples CDecision Tables & Calculators DNumbers Provenance & Forecast Register EGlossary, Phase-Gate Timeline & Learning/Community Map FFailure-Mode / FMEA Catalog GRegional & International Design Deltas: Consolidated Quick-Reference Crosswalk

Guide › Glossary › Power capping

Power capping

Limiting how much power chips or racks can draw to stay within facility limits, at the cost of some performance.

The Definitive Guide to AI Data Centers · 173 chapters · 2026Ask the Guide · Calculators · Register · Glossary · Feedback