Inference engineering for DeepSeek V4 Pro 0813

This post details the inference engineering work done to create a production-ready day zero API for DeepSeek V4 Pro 0813. It highlights adjustments made to the inference stack configurations (parallelism, KV cache allocation, prefill-decode worker ratio) based on evolved traffic patterns, longer sequence lengths, and higher cache hit rates. It also covers changes to the frontend and chat template to adhere to new harness expectations and API specifications, and the integration of DSpark speculator for speculative decoding to improve TPS.