failure rate, top 5% most resource-intensive large-LLM tasks (Alibaba Unicron, Dec 2023); ~73% of failures fixable by restarting the job
~43.4%observed
| Value kind | observed — Reported measurements, counts, and specifications keep the precision and scope stated by their source; an exact specification is not treated as a range. |
|---|---|
| Scope | Failure (abnormal-termination) rate of the top 5% most resource-intensive large-LLM training tasks in Alibaba's production fleet, as reported in the Unicron paper (arXiv, December 2023). It is not a per-job, per-GPU or fleet-wide rate. The ~73% share of failures fixable by restarting the job is the paper's own error-analysis figure, not a hardware-attribution share. The paper does not establish device-population exposure, so do not annualize this into a component failure rate or apply it to another operator's job mix. |
| As of | 2023-12 |
| Source | Unicron (He et al., arXiv 2401.00134) |
| Review | checking…review by 2027-04-16 · fast cadence |
| Recorded changes | last 2026-10-16 · 3 revisions tracked |
| Claim id | large-llm-job-failure-rate-top-5-most-resource |
Where the guide uses it
- 10.7 Fleet Reliability, Fault Tolerance & Autonomous Recovery
- 12.2 The AI-Cluster Reliability Rethink: Goodput vs Facility Availability
← Full numbers register — every date-stamped figure in the guide, with revision history.