Software, Orchestration & Service Delivery

The scheduler decides whether your GPUs run jobs or sit idle. Choosing Slurm, Kubernetes or both is a bet on your workload mix and on what it costs to operate two control planes.

For tightly coupled groups, placement sets performance: inside one NVLink domain the job runs at scale-up bandwidth, and split across the boundary it falls off a bandwidth cliff no tuning recovers. Price that loss against the idle capacity that keeps the group local.

Sharing a GPU takes two separate choices: how the silicon is divided (whole GPU, MIG, MPS, time-slicing, fractional) and how tenants are walled off (process, container, VM, confidential VM). Conflate them and you sell a partition as a security boundary.

Driver, CUDA/ROCm, NCCL/RCCL and firmware behave as one coupled tuple. Pin and attest the exact artifacts, test that they work together, and admit only a tested tuple at each scheduling boundary. One untested mix can hang a run or silently slow it.

Declarative, pipeline-driven provisioning decides how many GPU-hours a fleet burns between power-on and the first paid job: at initial bring-up, at every refresh and at every node failure.

GPU-fleet observability protects goodput. What you detect, how fast you attribute it, and which telemetry you can afford to keep decide whether the cluster keeps training or stops to restart.

Job-level MTBF collapses as nodes are added, so training reliability is a control-plane problem: detect, fence the failed rank, select a complete surviving checkpoint, verify the restarted step. Measure lost progress and the whole recovery interval; every minute of it is lost goodput.

Framework choice, parallelism layout, orchestration and checkpointing decide how much of a fleet's paid arithmetic becomes trained weights. A wrong layout can idle half the FLOPS at full price.

A cluster becomes a product when customers can buy it, run a job, be metered and leave. The rung you sell on the bare-metal-to-serverless ladder sets your isolation, SLA exposure, billing and margin.

Data rights can force a retrain, a model withdrawal or an in-region deployment. One 2025 discovery order, affirmed in January 2026, put 20 million de-identified conversation logs in the opposing side's hands. Design lineage and residency before a corpus or hall becomes an unusable investment.

Inference serving exists to maximize goodput: tokens from requests that met their quality and latency limits. Batching, chunking, disaggregation, speculation and routing each bet on where that optimum sits for your model and traffic; chase raw throughput instead and you ship deadline misses.