Quantization
Using lower numerical precision (FP8, FP4, INT8) to cut memory and boost throughput, trading some accuracy for speed.
Current numbers
93.3% (~15x)DeepSeek-V2 reported a 93.3% KV-cache reduction (~15×) versus DeepSeek-67B, from MLA plus KV quantization