How to build a day-0 API for Kimi K3

What this post added

This post describes the technical work involved in enabling day-0 support for the Kimi K3 model on Baseten's Model APIs. It details the challenges and solutions for running a 2.8T parameter model with novel architectural features like KDA, AttnRes, and Stable LatentMoE. Specific contributions include integrating with vLLM and SGLang, validating model fidelity with Kimi Vendor Verifier, optimizing inference configurations, developing a custom tokenizer that is up to 18x faster than tiktoken for long sequences, and implementing performance enhancements like speculative decoding and disaggregation of prefill and decode stages.