GPU Confidential Computing & Trusted Execution

Confidential computing answers a question that other controls cannot answer alone: how do you run on infrastructure you do not trust, and prove it? The tenant renting a GPU in a multi-tenant cloud, the sovereign government placing a frontier model in a hyperscaler region, and the healthcare or finance customer whose data is the product all face the same choice. They can take the operator's word that no privileged insider, compromised hypervisor or co-resident tenant can read their weights and prompts. Or they can demand a cryptographic guarantee rooted in silicon, enforced by hardware the operator cannot override from software, and attested end to end before a single byte of plaintext is released.

That guarantee is a Trusted Execution Environment (TEE), at once a strong cryptographic primitive and a leaky abstraction. On a supported CPU/GPU pair, turn it on and the operator's software can no longer read your data in use, the hardest of the three data states to protect. Turn it on without knowing what it does not cover and you have bought a false sense of security. Plaintext metadata still leaks, side channels still exist, the operator still controls scheduling and can deny service, and anyone who can touch the server can attack its memory bus. The whole chain also rests on a firmware root of trust that someone has to attest. Chapter 10.3 (the sharing and resource-allocation model) and Chapter 11.6 (multi-tenant isolation) both point here.

Data exists in three states, and the security maturity of each differs widely. At rest (weights on NVMe, checkpoints in object storage) is solved: AES-256 self-encrypting drives and KMS-wrapped keys are table stakes. OCP L.O.C.K. (1.0, September 2025; 1.1 at release candidate, June 2026) makes Caliptra in the SSD controller the only component that can read media encryption keys, which is designed to keep a drive-firmware bug from exposing them. Google and Microsoft have committed to it for their storage products, so ask SSD suppliers when they will support it. In transit (across the fabric, between nodes) is solved too, with TLS, IPsec and increasingly link-layer encryption such as MACsec and NVLink encryption. In use is the moment the weights are decrypted into GPU HBM and the prompts flow through the tensor cores. Until recently that state was simply unprotected. A privileged host with root, a malicious hypervisor, a DMA-capable peripheral or a cold-boot attacker could read plaintext model and data straight out of memory. Confidential computing exists to address that last gap, and only that gap. Chapter 11.8 gives the full at-rest, in-transit and in-use treatment for weights.

The mechanism is a hardware-enforced boundary. A CPU TEE (a confidential VM) encrypts guest memory with a per-VM key held in the memory controller that no hypervisor or co-tenant can extract, and hardware access checks stop the hypervisor from remapping or replaying its pages in software. The memory encryption itself is deterministic and has no replay protection, which is what physical bus attacks exploit (see residual attack surface below). A GPU TEE protects HBM with hardware access control and hides most of the GPU's register and command interface from the host. The two are chained: the confidential VM and the confidential GPU context set up an encrypted, mutually attested channel, so the bounce buffers that carry data across the untrusted PCIe bus are themselves encrypted. For an AI data center the model runs on the GPU, so a CPU TEE alone protects nothing that matters. You need both, chained and attested, and that chaining is where the cost and the caveats live.

The GPU TEE does not stand alone. It is anchored to a CPU confidential VM, and your host platform sets which one you get; you do not choose freely per workload. The two production options differ in design, and that difference cascades into attestation tooling and cloud availability.

AMD SEV-SNP (Secure Encrypted Virtualization with Secure Nested Paging) encrypts each VM's memory with a per-VM key managed by the on-die AMD Secure Processor. Secure Nested Paging stops the hypervisor from remapping, replaying or aliasing guest pages, which fixes the integrity gap that earlier SEV generations left open. Its boundary is per-VM and relatively coarse. Intel TDX (Trust Domain Extensions) builds a "trust domain" enforced by a dedicated, Intel-signed firmware module, the TDX Module, running in a new SEAM mode and mediating every transition between the untrusted hypervisor and the protected guest. TDX leans on a measured, attestable firmware mediator; SEV-SNP leans on the Secure Processor. Both produce a signed attestation report of the confidential VM's launch measurement.

The practical consequence is that your accelerator host CPU determines your TEE: an AMD EPYC host pairs with SEV-SNP, an Intel Xeon host with TDX. The GPU confidential-computing stack, the attestation verifier and the cloud regions where it is GA all follow from that one platform decision.

Grace-hosted NVL72 racks fall outside both. NVIDIA’s confidential-computing deployment guide (v7.1, April 2026) lists only x86 hosts: Intel TDX on Emerald Rapids or Granite Rapids, and AMD SEV-SNP on Milan, Genoa or Turin, with Hopper, B200 or RTX PRO 6000 GPUs. The Arm-based Grace CPU in GB200 and GB300 NVL72 racks has neither SEV-SNP nor TDX, and NVIDIA documents no Arm CCA confidential VM for it. Every measured result in this chapter ran on x86 hosts. If your fleet is NVL72, get NVIDIA’s current support statement for Grace before you promise a tenant confidential computing. Vera Rubin’s rack-scale claim is covered below.

Making a GPU confidential is a different engineering problem from making a CPU confidential, because a GPU is a wide-open device by design. It has hundreds of memory-mapped registers, DMA engines that move data on their own, and a command interface the driver pokes directly. An independent teardown of NVIDIA's implementation (arXiv 2507.02770, alongside NVIDIA's own WP-12554) shows three load-bearing mechanisms.

The Compute Protected Region (CPR) carves off the large majority of GPU memory, roughly 90% of HBM, into a region under hardware access control that only the confidential GPU context can read. HBM is access-controlled, not encrypted. Code and data sit in plaintext in HBM inside the GPU package, and no NVIDIA primary source says Blackwell encrypts HBM. What is encrypted is the traffic that crosses the untrusted host boundary and, on Blackwell, NVLink. Weights and activations live in the CPR, and the host is denied access. The BAR0 decoupler handles register exposure. In confidential-compute mode about 99.78% of register fields read back as zero through the decoupler, shrinking the management interface the host can touch to a tiny, audited surface. Encrypted transfers protect the data path. PCIe is untrusted, so every payload crossing it moves through AES-GCM-encrypted bounce or staging buffers, with per-channel keys derived from an SPDM-negotiated master secret (Gu et al.’s Appendix B enumerates 44 derived keys across RPC, DMA, fault and workload channels in the tested Hopper implementation). The result is a GPU whose memory and management plane the operator cannot read through the host, even with full host root.

The most consequential row in that table is GPU scope. Hopper's supported multi-GPU mode passes all eight GPUs of the documented four-NVSwitch, air-cooled HGX system into one confidential VM, but GPU-to-GPU NVLink traffic stays unencrypted. Blackwell encrypts peer-to-peer NVLink traffic for up to eight GPUs in one CVM. Choose Blackwell when the confidentiality boundary must include intra-node scale-up traffic, and remember that the documented scope is still one eight-GPU CVM, not a rack-scale domain. Every mode NVIDIA documents passes whole GPUs to a confidential VM, one or up to eight at a time, and never shares a GPU among VMs (NVIDIA Secure AI whitepaper, August 2025; R595 release notes, April 2026). No documented mode puts a MIG slice or a vGPU inside a TEE, so a confidential tier is sold by the whole GPU (Chapter 11.6 prices it).

NVIDIA markets Vera Rubin NVL72 (shipping since June 2026) with third-generation confidential computing that it says extends across the full rack-scale platform: CPUs, GPUs and the NVLink fabric. That is still a vendor announcement as of October 2026. We found no public release notes, support matrix or independent measurement showing a 72-GPU confidential VM in operation. Before planning beyond eight GPUs, ask NVIDIA for the driver release and the supported-configuration matrix.

A production example was announced in June 2026, when Apple extended Private Cloud Compute onto Google Cloud. It runs Apple Intelligence workloads on NVIDIA confidential computing (NVIDIA GPUs) with Intel TDX CPUs and Google's Titan root of trust. It combines software attestation anchored in at least two independent vendors' roots of trust, a record of every PCC machine in the fleet kept in a cryptographically verifiable append-only ledger, and a commitment to publish its binaries (Apple SEAR, 2026-06-08). Apple calls the Google Cloud deployment a summer 2026 preview that ramps toward the full PCC protections. Confidential inference on third-party GPUs is no longer only research: it is part of a consumer product's production design.

In June 2026 (revised in July), Phala, a confidential-computing provider, published a B300 study (arXiv 2606.23969) that ran GPU confidential computing under Intel TDX. B300 is missing from the support table in NVIDIA’s v7.1 deployment guide, but NVIDIA’s R595 release notes (April 2026) list HGX B300 for both the single-GPU and the multi-GPU confidential modes. BF16 matmul ran at 0.998× of native, yet dense and MoE LLM serving lost 13–27% of throughput and model loading ran 34× slower. The authors' scheduling fixes recovered most of that gap. The loss comes from the path between the confidential VM and the GPU, not from the tensor cores.

The other large accelerator platforms, AMD Instinct, Google TPU and AWS Trainium, document no confidential mode that a tenant can turn on and attest as of October 2026. Google Cloud’s Confidential VM documentation (October 2026) says TPUs cannot be attached to a Confidential VM. Google’s own Private AI Compute serves Gemini on what its November 2025 technical brief calls a hardened TPU platform with properties “comparable to a typical TEE”, but only for Google’s own services. AWS states that no mechanism exists for its operators to reach data sent to an accelerator on a Nitro instance, and attests the instance through NitroTPM, yet it documents no TEE on Trainium itself. That guarantee rests on AWS’s system design, not on evidence the accelerator produces. AMD markets its Helios rack (MI455X, volume deployments expected in the second half of 2026) with hardware-enforced isolation, device identity and encrypted memory and interconnects. Like Vera Rubin’s rack-scale claim, it is a vendor statement; we found no AMD release notes or support matrix for a confidential Instinct mode. Among these platforms, a tenant who must verify the accelerator itself, not just the host, has only NVIDIA’s documented modes.

Encryption protects data on the path it covers, but releasing a key remotely also needs proof that the recipient is the confidential environment you intended. A TEE you cannot prove is genuine, running the firmware and configuration you expect, is just a black box asserting that it is trustworthy. Attestation is the cryptographic proof, and it is operationally demanding because it puts a hard dependency in your boot path that did not exist before.

In NVIDIA's implementation the flow runs roughly as follows. The GPU presents a device identity chain (the studied implementation reports a 5-certificate chain rooted in a vendor-provisioned device identity key) and a set of measurements, 64 records in Gu et al.’s tested Hopper implementation (3 July 2025, §8), that capture firmware versions, configuration and security state. A verifier (NVIDIA's Remote Attestation Service, NRAS, or a self-hosted equivalent) checks the chain's signatures and compares each measurement with the Reference Integrity Manifest (RIM), the vendor-published golden measurements for that exact firmware build. Only if the evidence passes identity, authenticity, freshness, revocation and approved-state policy does the verifier issue an attestation token. The relying party, the key broker, accepts that token as the precondition for releasing the decryption keys that let the workload start. The CPU TEE goes through the analogous SEV-SNP or TDX attestation against its own verifier.

Before releasing workload secrets, bind both the CPU/CVM and every assigned GPU to the tenant, the workload image and configuration, a fresh challenge and the recipient key. The record count is not a completeness score. The weak joint is the link between GPU and CVM: NVIDIA GPU attestation reports are not bound to any particular CVM or CPU. TEE.fail exploited this: its authors combined a forged CPU quote with a real GPU report taken from another machine. Judging the CPU evidence and the GPU evidence separately, then accepting both, is not enough. Match the GPU identities in the evidence against the devices you recorded for that server at acceptance (11.4). Then ask the platform vendor how its verifier links the GPU's evidence to the attested CVM. A forged CPU quote defeats any such binding, so physical custody (see the residual attack surface below) belongs inside the boundary.

Confidential GPU topology must match the documented CVM boundary. Both generations stop at eight GPUs per CVM; the difference is whether NVLink traffic inside it is encrypted. The decision is whether your workload's confidentiality boundary must include intra-node scale-up traffic, not whether Hopper is limited to one GPU. Confirm your platform and mode against the vendor's support matrix before you price either.

Blackwell adds two separate protections. Encrypted NVLink is the first: it protects peer-to-peer traffic inside one CVM of up to eight GPUs. Collectives therefore stay within the TEE and skip the PCIe bounce buffers. TEE-I/O (PCIe TDISP with link encryption) is a different feature. It would let the GPU write directly into confidential-VM memory and drop the software bounce buffer. It needs a TEE-I/O host, such as Intel TDX Connect, which ships on some Xeon 6 parts, or AMD SEV-TIO. NVIDIA documents no production configuration that uses it as of October 2026, and its September 2026 production report still routes host-to-device traffic through encrypted bounce buffers. Encrypted collectives cost something too: in a paired-run B200 study, eight-GPU training lost ~10–13% (serving results are in the final section, When to turn it on).

The link to the neighboring chapter is structural. The same NVLink scale-up domain that Chapter 11.6 analyzes as an isolation boundary between tenants is what encrypted NVLink turns into a confidentiality boundary against the operator. The two chapters describe the same fabric under two different threat models.

Hardware TEEs are necessary but not sufficient, because they protect memory, not the software lifecycle around it. Two pieces of plumbing turn a confidential VM into a usable confidential workload, and each carries its own decision.

Confidential containers (the Kata/CoCo lineage and its cloud equivalents) run each pod or container inside its own confidential VM. That moves the orchestrator (Kubernetes, the node agent, the cloud control plane) outside the trust boundary. In a normal Kubernetes node the kubelet and container runtime can read any container's memory; confidential containers break that. The cost is a fatter per-pod footprint, a VM per workload instead of a namespace, plus an attestation step in pod startup. The decision is granularity: a confidential VM per node is cheaper but coarser, and a confidential container per workload gives stronger isolation at more weight.

The key broker is the linchpin and the most common place to get the architecture subtly wrong. The pattern is attestation-gated key release. Encrypted weights and data sit at rest. The workload boots into its TEE and presents its attestation token to a key broker service. The broker validates the token (right firmware, right measurements, right identity) and only then releases the decryption keys into the attested enclave, where the operator cannot see them. The weights never exist in plaintext outside a proven-good TEE.

Get this right and you have a strong story. Release keys before validating attestation, or run the broker somewhere the operator can read its memory, and you have moved the crown jewels' keys into the environment you were trying to defend against. Attestation proves genuine hardware in a measured state; it does not vouch for the code that receives the plaintext. The release policy therefore has to name the approved image, configuration, workload identity, recipient key, device set and evidence freshness. A vulnerable or malicious approved workload discloses data from inside a perfectly valid TEE, and the vendor's threat model excludes compromised workload software. Yours should too. Chapter 11.8 relies on this attestation-gated release for in-use weight protection and holds the key-management discipline (HSM/KMS hierarchy, rotation, revocation).

Confidential computing is a strong primitive with a tightly bounded threat model, and in practice the failure is almost never the cryptography. It is a defender who assumed the boundary covered more than it does. The literature on where confidential VMs fall short (arXiv 2503.08256, a systematization of CVM trust relationships) is required reading before you stake a sovereignty claim on a TEE.

• Metadata leakage. RPC payloads are encrypted, but queue headers, command-ring structures and physical-address tables on the untrusted side remain plaintext. An operator who cannot read your data may still infer access patterns, memory layout and operation sequencing from that metadata. For some workloads that is harmless. For others, access-pattern leakage is itself a meaningful side channel.
• Uncore and microarchitectural side channels. The TEE protects the compute and memory of the protected context, but shared uncore blocks (media engines such as NVENC/NVDEC/NVJPEG, DRAM frequency scaling, contention on shared caches and interconnects) can leak information across the boundary. Chapter 11.6 documents the same class of channels bypassing MIG and MPS isolation, and a TEE does not make them disappear.
• The trusted-but-unverified scheduler. The hypervisor leaves the confidentiality boundary but stays in the availability and scheduling path. It still decides when your VM runs, can starve it, and can mount denial of service. In some designs the firmware mediator (the TDX Module, the Secure Processor) is also a large trusted-computing-base component that you attest but do not audit.
• The firmware / RIM supply chain. The whole guarantee bottoms out in golden measurements published by the vendor and a firmware root of trust on the device. If the RIM is wrong, the firmware signing is compromised or the silicon root of trust is subverted, attestation passes for a compromised platform. That is why Chapter 11.4 (hardware root of trust, Caliptra/DICE, OCP S.A.F.E.) is the foundation under this chapter: the TEE is only as trustworthy as the firmware integrity beneath it.
• Physical access to the host. A CPU TEE does not defend the DDR5 memory bus. Intel and AMD both treat physical memory attacks as out of scope, and AMD has said it plans no mitigation (bulletin AMD-SB-3040). Researchers have shown that a cheap interposer sitting between a memory module and its slot, combined with control of the host that an operator already has, is enough to break the confidentiality and attestation guarantees of a CPU TEE. The attack works because the memory encryption is deterministic. The same data at the same address always produces the same ciphertext, so an interposer can match encrypted blocks, and nothing detects an old ciphertext written back in place. TEE.fail (IEEE S&P 2026, disclosed October 2025) needed under $1,000 of parts. It recovered Intel's TDX attestation key and used it to forge quotes that verified as "UpToDate". It also pulled keys out of an SEV-SNP guest, and it passed NVIDIA GPU attestation while the workload ran outside the TEE. DDRop, disclosed on 14 September 2026 for ACM CCS 2026, is cheaper still at about $159 of parts per unit. It forged TDX launch measurements and copied SEV-SNP guest pages, but only once the attacker controlled the host and could fit the device. The GPU TEE hangs off the CPU confidential VM, so a forged or compromised CVM weakens the GPU's protection as well. So a TEE shuts the operator out only while the operator has no physical access to the server. Write that into the contract: named custody of each server, tamper-evident seals, logged access and two-person rules in the hall. Without those terms, count the operator as still inside the boundary. Advanced fault injection and supply-chain tampering before provisioning also fall outside the standard confidential-computing threat model.

Confidential computing is the right tool when a specific, unavoidable, untrusted party sits in your data path and you cannot otherwise prevent plaintext exposure of the workload. Before you pay for it, ask whether moving the data to a holder you trust, or to another site, would remove that party more cheaply (the callout at the top of the chapter sets out the test). On Blackwell, encrypted NVLink extends the protection to GPU-to-GPU traffic, and your configuration and workload decide what that costs.

A 2026 paired-run B200 benchmark puts a correctly configured confidential serving stack at ~1–3%, a stock stack at 30–40% and eight-GPU confidential training at ~10–13%. The cost sits mostly in encrypted collectives and host-boundary crossings. Matrix-multiply (GEMM) throughput barely moves, but a B300 harness run by Phala clocked a memory-bound kernel at ~0.91× of native. "Correctly configured" is concrete, and NVIDIA's own September 2026 report shows what it takes. On eight B200s running DeepSeek-R1 under TensorRT-LLM (a vendor result on one model), throughput stayed at 96–98% of non-confidential and per-token latency rose only 1–4%. Getting there took three framework changes. Bounce-buffer paths use pageable rather than pinned memory, token readback runs asynchronously, and the autotuner uses a different timer. B200 confidential mode also lacks NVLink SHARP multicast, so collective libraries must detect that and pick other algorithms. Ask your serving-stack vendor which of the three its release includes.

These are one-host measurements. Publish the scope of any benchmark you cite, and make an acceptance test on your own workload the criterion: platform, enabled I/O path, software version and transfer pattern set the number, not the generation label. The cost covers both cycles and operations. You now run an attestation verifier and RIM pipeline as Tier-0 infrastructure, you architect a key broker that must never leak, and you carry a documented residual surface of metadata, side channels, a trusted scheduler, a firmware supply chain and physical custody of the host. Defend those with the rest of this Part rather than assuming the TEE covered them.