H100
NVIDIA's Hopper-generation data-center GPU; the chip behind most of the 2023–2024 buildout wave and the baseline many density and power comparisons start from.
Current numbers
~$16–27kused H100 secondary price by form factor (PCIe ~$16–20k; SXM5 ~$22–27k); no reliable age-stratified curve published
Measure resume, model-ready and TTFT separatelyserverless GPU time-to-first-token (H100): warm-pool vs scale-from-zero; snapshots claim ~10x cold-start gains
80 → 288 GB → ~1 TBHBM capacity per accelerator package: H100 80 GB; B200 180 GB; B300 288 GB (HBM3E); Rubin Ultra ~1 TB HBM4E on a four-chiplet package (roadmap)