The efficient frontier of LLM inference

This post details inference engineering techniques for LLMs, categorizing them into those that manage tradeoffs (batch sizing, parallelism strategy, quantization) and those that move the entire performance frontier (kernel optimization, speculative decoding, disaggregation). It specifically discusses how these techniques apply to models like GLM-5.3 and Kimi K3 in agentic coding scenarios with KV cache reuse, providing a framework for understanding and optimizing LLM inference efficiency.