SGLang
An open-source LLM serving framework with fast structured generation and aggressive KV-cache reuse.
Current numbers
~29% lower (B200 1.41x)MI355X (ROCm, SGLang, FP8) cost per million tokens vs B200, GLM-5 on one node, best point, May 2026
An open-source LLM serving framework with fast structured generation and aggressive KV-cache reuse.