FP16 · 16-bit floating point
A half-precision number format; it is the usual reference rung of the precision ladder, since each step down from it roughly doubles a chip's headline throughput.
Current numbers
~$1.90/M tokCost per million tokens, Spheron: Llama-3.1-70B FP16 on 8×H100 at $2.40/GPU-hr, ~2,800 tok/s (vLLM, batch 256, 512-in/512-out)