unplanned interruption rate, Llama 3 405B pre-training on up to 16,384 H100s: 419 unexpected of 466 total interruptions over a 54-day window (54 × 24 / 419 ≈ 3.1 h); >90% effective training time with automation

Name: unplanned interruption rate, Llama 3 405B pre-training on up to 16,384 H100s: 419 unexpected of 466 total interruptions over a 54-day window (54 × 24 / 419 ≈ 3.1 h); >90% effective training time with…
License: https://aidatacenterguide.com/terms#intellectual-property

1 every ~3 hr (419 in 54 days)observed

Value kind	observed — Reported measurements, counts, and specifications keep the precision and scope stated by their source; an exact specification is not treated as a range.
Scope	Whole-job interruptions for one run under Meta's event definition; not a per-GPU MTBF, an equipment AFR, facility availability or a scaling law for another fleet size.
Caveat	Table 5 prints 30.1% beside its 148 faulty-GPU events, but 148 of 419 is 35.3% (the 72 HBM3 events are 17.2%), so the paper's 58.7% GPU share, the sum of the printed percentages, understates the counted 64% (268 of 419). Use the counts and state your denominator.
As of	2024-07
Source	Meta (Llama 3 Herd of Models) · The Llama 3 Herd of Models, §3.3.4 and Table 5
Review	checking…review by 2027-07-23 · standard cadence
Recorded changes	last 2026-10-19 · 2 revisions tracked
Claim id	unplanned-interruption-rate-llama-3-405b-419

Where the guide uses it

← Full numbers register — every date-stamped figure in the guide, with revision history.