Agentic Ops, RL Control & the Autonomy Ladder

An AI campus runs thousands of coupled control loops, at a tempo and dimensionality no human shift can track, alongside established PLC loops, protective relays and automated recovery. This Part has also assigned human responsibilities: the operator who acknowledges an alarm, the technician who replaces a lemon node, the change board that approves the change and the switching authority that authorizes its execution. This chapter takes up the decision to hand some of that work to software that acts. Such software moves a setpoint, sheds a workload or fails over a CDU without a person in the loop, rather than only observing or recommending. The economics argue for it: the gap between a well-tuned plant and a conservatively tuned one shows up directly in PUE, in goodput and in megawatts you do not have to procure. The failure modes argue for caution. An autonomous controller does not fail like a tired operator. It fails faster, in correlated ways, and with worse attribution.

The organizing device is the autonomy ladder: a few named rungs, each with an explicit promotion gate. That turns "how autonomous is this facility" from a marketing adjective into an auditable property of each loop. The chapter works through the ladder, then the engineering that makes the top rungs safe (reinforcement-learning joint control bounded by a hard safety envelope), then the oversight and liability boundary that decides when an agent may act unsupervised at all. This is the consolidating chapter for autonomy. The underlying control physics, telemetry, recovery and design-time twins live elsewhere and are cross-referenced, not re-derived.

The most common scoping error is treating autonomy as a facility-level property: asking "should the facility be autonomous?" instead of "which loop, at which rung, gated by what?" The ladder has four rungs, and the load-bearing engineering sits in the gate between two rungs, the evidence you must produce before a loop may climb.

Rung 0 — Observe. The system ingests telemetry and renders state: dashboards, trends, anomaly flags. The new agent has no actuator authority. Established PLC loops, local controllers, automatic transfer and protective trips keep their commissioned roles. The agent's observation depends on the telemetry and observability stack: garbage in, autonomous garbage out. Chapter 14.2 builds this rung.

Rung 1 — Recommend. The system proposes an action and a human executes it: "shift this training job to the cooler hall," "raise loop supply temperature 1.5 °C," "this CDU pump is trending to failure, schedule it." The agent's judgment is on trial but its hands are tied. The human evaluates each recommendation, independent interlocks and protection keep their authority, and contracts assign accountability. Judge a predictive recommender on how many of its warnings are real failures, how much warning it gives and what a false drain costs, not on raw accuracy (published fleet results are in Chapter 14.3). Most 2026 production deployments sit here, and it is a defensible place to stay for any loop whose failure mode is catastrophic.

Rung 2 — Act within an envelope. The agent executes directly, but only inside a pre-declared, hard-bounded action space, and a human supervises on-the-loop: watching, able to intervene, but not approving each action. Most of the economic value sits here: closed-loop cooling optimization, workload-aware power capping, automated node ejection and re-balancing. The safety envelope (next section) does its real work at this rung, because it is all that stands between the agent and a constraint violation.

Rung 3 — Fully autonomous. The agent owns the loop end to end, including the decision to escalate to a human, and operates out-of-the-loop for routine action. In 2026 this rung is reserved for narrowly scoped, low-consequence, high-frequency loops (fan-speed trim, individual-rack power-state management, rollout-pool autoscaling), and it stays the exception. A facility that claims Rung 3 across the board in 2026 is either lying or has not yet had its first correlated failure.

Trace and outcome. In the worst stale-reading case, a frozen flow value keeps steering the optimizer until the shield marks it stale at 5 s, and the fallback takes over 3 s later. That is 8 s of exposure against a 20 s thermal deadline, a 12 s margin, so the design passes on paper. It is still only a design figure: the stale-sensor and handover tests have not run, so the loop stays at Rung 1. An operator evaluates each recommendation, local control stays in charge, and the optimizer gets no actuator credentials. The cost is the optimization benefit you defer. What you avoid is an unbounded pump action on flow data nobody has checked.

Flip. Promote only when every release-gate item passes, actuator limits and recovery included, and end-to-end tests put measured staleness detection plus handover inside the deadline with margin. The same tests must cover a stuck sensor that keeps sending fresh timestamps, caught by cross-checking pump speed and differential pressure, and the results go in the loop register. The gate fails once the measured total reaches the deadline: at these numbers, a handover of 15 s or slower, or a rack whose deadline is 8 s or less. A later stale-data or model-validity breach demotes the loop without touching local protection. DeepMind's 2018 autonomous cooling system relied on the same two safeguards: a narrowed operating range, plus a separate rule-based controller to fall back on (DeepMind’s 2018 account). Chapter 14.2 covers measurement quality and Chapter 14.7 the response-time budget.

The reason autonomy is worth the risk is joint optimization. Workload placement, cooling and power are physically coupled. A training job that shifts its all-reduce phase changes rack power within seconds, which changes heat load, which changes the cooling plant's operating point, which changes facility draw and therefore the power chain's losses. Run three independent control loops and they fight at the margins: the cooling loop chases a temperature the workload loop just invalidated, and the power loop caps a rack the cooling loop was about to cool efficiently. A single agent that sees all three state spaces can find an operating point none of the three can find alone. RL is the natural formalism: sequential control of a coupled physical system with a clear reward (energy, or goodput-per-watt) and hard constraints, which is the core of ML-driven cooling.

By 2026 the historical DeepMind proof point has commercial successors. Since 2025, vendors have been testing and fielding software that acts on the plant and the load itself. Phaidra is a partner in NVIDIA's Omniverse DSX blueprint. Phaidra said in November 2025 that its liquid-cooling control agent had been tested on production DGX GB200 systems while they ran live AI workloads. In September 2026 Aligned Data Centers and Phaidra announced a rollout of Prism, a read-only operations layer. That puts the disclosed customer deployment at Rung 0–1. Operators have not followed the vendors up the ladder: Uptime's 2026 survey finds trust highest for predictive maintenance and sensor analytics, not autonomous control.

Grid flexibility is the other live case. Emerald AI's Conductor already curtails a production AI factory when the utility signals. NVIDIA says Conductor will integrate into its DSX Flex grid-flexibility platform. On an August 2026 evening, a Silicon Valley Power signal took NVIDIA's Eos factory in Santa Clara from 4 MW to 3 MW. The lowest-priority jobs paused while high-priority inference kept running. NVIDIA says Conductor answers such signals in under a minute and has handled more than 200 more correctly since (vendor-reported). By NVIDIA's own account, Santa Clara is not a DSX Flex installation. NVIDIA has named Manassas, Virginia, as its first dedicated DSX Flex site, a Vera Rubin AI factory of 96 MW that follows five earlier demonstrations. Emerald AI said in June that it was planned for later in 2026; NVIDIA's September post gives no date. Automation like this is Rung 2 work. It must not drop load on its own during a grid disturbance. ERCOT (NOGRR282) and SPP have ride-through rules in force for new large loads, and PJM proposed its own requirements in September 2026. Chapter 4.10 covers the rules and Chapter 14.7 the operating response.

The canonical proof point is Google DeepMind's data-center cooling work. In a 2016 live test, a system trained on historical sensor data cut cooling energy by ~40%, equal to a ~15% cut in overall PUE overhead, using its recommended actions. The 2018 version took direct control: it read thousands of sensors every five minutes and adjusted chillers and pumps below the conservative setpoints a human would hold. It started at about 12% savings and averaged about 30% after nine months, inside operator-set constraints with automatic fallback to rule-based control (DeepMind/Google, 2016 and 2018). A separate 2022 real-world demonstration with Trane Technologies delivered ~9% and ~13% energy savings at two live commercial buildings, against heuristic controllers (DeepMind, arXiv:2211.07357, 2022). Its production obstacles were evaluation under non-stationarity, learning from logged offline data and constraint satisfaction. They show where the work is: the optimizer is the easy part, and bounding it safely in a live plant is the engineering.

The safety envelope is what makes RL deployable on a plant that can destroy hardware. It is not a soft penalty term in the reward: a learned policy will trade a small constraint violation for a large reward if you let it, which is exactly the behavior you cannot tolerate near a Pegatron VR200 NVL72 rack at 228 kW Max P. The envelope is a hard outer loop that the agent cannot override:

• Constraint shielding. Every proposed action is checked against hard limits before it reaches an actuator. The limits cover coolant inside the platform's OEM-approved operating point (fluid, inlet setpoint, liquid heat load, design ΔT, flow, approach, pressure and transient margin), breaker and PDU limits, and ramp-rate limits on cooling actuators. For the QCT GB200 NVL72 example, 45 °C maximum liquid inlet and 65 °C maximum liquid return are independent acceptance limits, not a selected 20 K operating point. An action that would violate a limit is rejected and replaced by a known-safe fallback, however good the agent thinks it is.
• A conservative fallback controller. The classical rule-based or PID controller is not decommissioned when the RL agent goes live. It becomes the floor. If the agent's policy is uncertain, its inputs go stale or it is throttled, control reverts to the deterministic controller the plant was commissioned against.
• Watchdogs and rate limits. The agent acts at a bounded cadence with bounded step sizes, so a single bad inference cannot slam a setpoint. A watchdog that detects the agent stalling or oscillating trips control back to the fallback.

The design decision is where the envelope's walls sit and what lives outside them. Set the walls too tight and you have bought an expensive optimizer that only nibbles at the margins the conservative controller already left on the table. Set them too loose and the first distribution shift (a hot day, an unmodeled workload pattern, a failed sensor the agent trusts) walks the plant into a violation the envelope was supposed to catch. The walls are a thermal-and-electrical-margin decision, and they belong to the engineers who own the cooling and power chains. Cooling-control physics and setpoint strategy are in Chapter 15.2. The power-transient and ride-through limits that bound the electrical side are in Chapter 4.5.

The rung an action sits on is an engineering choice. The liability for that action is a legal and organizational one. Decide the two together, or you get the worst outcome: an agent empowered to act unsupervised under a contract that still pins the consequence on a human who was never in the loop. The oversight taxonomy is the bridge. In-the-loop: a human approves each action (Rung 1). On-the-loop: a human supervises and can intervene but does not approve each action (Rung 2). Out-of-the-loop: the agent acts and the human learns about it after the fact, if at all (Rung 3). The 2026 convention for critical infrastructure is blunt: out-of-the-loop is reserved for narrowly scoped, low-consequence actions, and most consequential actions run in- or on-the-loop under strict policy constraints.

Government guidance says the same. The December 2025 joint guidance on AI in operational technology from CISA and ASD's ACSC, with NSA, FBI and five allied agencies, tells operators to limit active AI control of OT without a human in the loop and to keep failsafes that revert to traditional automation or manual control. It adds that LLMs almost certainly should not make safety decisions for OT. It is guidance, not a regulation.

An agent may act unsupervised (out-of-the-loop) only when every condition holds. The action's worst-case consequence is bounded and recoverable. The action is frequent enough that human approval is genuinely infeasible (you cannot ask a person to approve a fan trim every five seconds). The failure mode has been analyzed for correlation, so a single bad decision cannot cascade across loops. And automatic limits and fallback, already tested, act inside the hazard’s deadline. When the event moves faster than a person can see, diagnose and act, human override is the backup, not the protection. Fail any one and the loop drops to on-the-loop at best.

Correlation is the hardest condition in practice. An agent making thousands of locally sound decisions can still drive a globally correlated failure. The canonical fear is a control agent that, responding to a real grid or thermal event, sheds or ramps load across the campus fast enough to become the NERC ride-through problem itself rather than its solution. The grid has already shown the scale of that problem: about 1.5 GW of data-center demand left the grid as sites transferred to backup during six faults over 82 seconds in Northern Virginia in July 2024, and about 3.8 GW did so on a single normally cleared 230 kV fault at Ashburn on 2026-07-22. Autonomy that can move power must be designed to not become a correlated load-shedding actuator.

Utility-signalled curtailment already runs without per-action approval. NVIDIA says no operator was required for the Eos shed described above: Conductor worked through a predefined workload hierarchy (NVIDIA, Sept 2026). It can run that way because the action is bounded: it has a fixed depth, a priority list set in advance, minute-scale ramps and a trigger from outside. That is Rung 2 as long as a human supervises and can intervene. Write the shed depth, ramp rate and priority order into the envelope specification. An agent able to shed a whole campus in under a second is a different action class, and it belongs in the last row of the table.

The liability column is the one a strategist must own. In a self-build, the operator owns the consequence of an agent it commissioned, and the envelope is its own design liability. In a colocation or neocloud arrangement, autonomy crosses an organizational boundary. If the operator's agentic control plane caps a tenant's racks to hold PUE, and that cap costs the tenant a checkpoint or an SLA, whose fault is it? Put an autonomy-declaration matrix in the agreement: every capability the control platform offers, the rung it operates at, the conditions under which the rung can change, and the rollback procedure if the human override fails. Without that matrix, autonomy is an un-priced, un-assigned risk sitting between two parties who each assume the other owns it. Chapter 14.11 covers the org that staffs the on-the-loop role and owns the goodput-versus-availability tradeoff, and, with Chapter 11.12, the converged cyber-physical escalation trigger in the table's last row.

Autonomy threads run through the whole guide, and the value of this chapter is partly that it refuses to re-litigate them. It keeps one home for "how autonomous, and how do we bound it," and points elsewhere for the substrate.

• Telemetry is the Rung 0 evidence floor, and an operational twin can supply a simulator for RL validation within its demonstrated model bounds. Chapter 14.2 builds both. The design-validation twin, which is distinct from the operational one, is in Chapter 2.7.
• Autonomous fault recovery (health checks, lemon-node ejection, automated break-fix) is a specific, mature instance of Rung 2/3 autonomy on the compute side. It lives in Chapter 10.7, so we do not re-derive it here.
• Energy-efficiency control (setpoints, free cooling, the ML-driven cooling optimization whose physics this chapter's RL agent acts on) is owned by Chapter 15.2.
• Transient absorption and closed-loop storage control (the BESS SoC loop, ride-through, power smoothing) is physics owned by Chapter 4.5.
• AI-assisted design uses ML to generate and validate the facility before it is built. That is a different activity and lives in Chapter 2.7; this chapter is strictly about run-time control of a commissioned plant.

The distinction that matters most: this chapter is not Chapter 14.11. That chapter automates the organization: runbooks, the human-error tradeoff, the shrinking operator bench. This chapter automates the plant: software acting on power, cooling and workload. The two meet at the on-the-loop human, which is why the liability table above hands the supervisory role back to the org chapter instead of inventing a new one.

A defensible autonomy program produces a small set of artifacts that make the ladder auditable, the same way Chapter 1.1 demands a design-basis document before steel is cut. Three are non-negotiable.

• The per-loop rung register. Every control loop, its current rung, its oversight posture, the gate it passed to get there, and its blast radius. This is the document you hand anyone who asks "how autonomous is the facility," and it forces the question down to a loop.
• The envelope specification. For every Rung 2+ loop: the hard constraints, the fallback controller, the watchdog triggers, and the designed safe state for the override-fails case. It is change-controlled under the Chapter 14.12 MOC process alongside the reward function.
• The autonomy-declaration matrix. Where autonomy crosses an organizational boundary (colo, neocloud, vendor control plane), the contractual mapping of capability → rung → rung-change conditions → rollback → liability owner. It turns un-priced risk into an assigned one.

The optimizer was never the hard part: cutting cooling energy 40% was demonstrated a decade ago. What keeps most consequential loops at Rung 1 or 2 in 2026 is bounding the optimizer so its failure modes are slower, smaller and more attributable than the human shift it replaces. These three artifacts are how a program shows, loop by gated loop, that it has done that.