How we built the new fastest API for GLM-5.2

This post details the technical optimizations made to create a significantly faster API for the GLM-5.2 model. Key contributions include: 1. Development of a 'GLM-5.2-Fast' API specifically tuned for reduced latency, trading off throughput. 2. Implementation of latency-focused parallelism strategies (Tensor and Expert Parallelism) for the fast API, distinct from the general API's Attention Data Parallelism. 3. A substantial reduction in the maximum batch size for the fast API to minimize resource contention. 4. General optimizations applied to both APIs, including scheduler improvements, updated NVFP4 weights, and an updated speculative decoding profile. 5. Bug fixes in the inference engine for both quality and performance. The post highlights the trade-off between latency and throughput, with the fast API having 50% higher input and output token prices due to its latency optimizations.