MFU · Model FLOPs Utilization
Model FLOPs achieved divided by the hardware's peak while a job runs. Meta reported 38–43% BF16 MFU training Llama 3 405B on 8,192–16,384 H100s; there is no fleet-wide operating band. Not a goodput measure.
How much of a chip's theoretical compute a training run actually uses; the headline training-efficiency metric.
Also written as: Model FLOPs Utilization
Current numbers
34% → 54%BF16 MFU on a 128-H100 cluster (GPT-3 175B) rose in 2024 from software and kernel maturation: same silicon, more throughput
~41%MFU against dense BF16 peak while pre-training Llama 3 405B on 16k H100s: a frontier-scale reference point
55.2% MFU (named MegaScale run)Training MFU on a named MegaScale run (ByteDance / Peking University); stricter than device-utilization figures usually quoted for enterprise clusters