How we built the world’s fastest API for GLM-5.2

This post details the specific optimizations implemented to achieve over 280 TPS for the GLM-5.2 model API. Key contributions include: implementing shared DSA for the GLM-5.2 architecture in the custom inference engine, performing in-house NVFP4 quantization from FP8 weights with equivalent quality on agentic benchmarks, implementing KV-aware routing with NVIDIA Dynamo for improved TTFT and reduced prefill burden, achieving a 2x higher TPS via disaggregated prefill and decode engines using NVIDIA Dynamo, and further improving TPS with GLM-5.2's Multi-Token Prediction heads for speculation.