Deploy Kimi K3 Dedicated Inference on CoreWeave

This post details the deployment of Kimi K3, a 2.8 trillion-parameter open-source model, on CoreWeave's Dedicated Inference platform using NVIDIA GB300 NVL72 systems. It provides performance benchmarks comparing GB300 and GB200 NVL72 for Kimi K3, focusing on output token throughput, mean request latency, and p50 decode throughput per user under agentic workloads. The post also describes the underlying stack, including NVIDIA Dynamo and vLLM with specific optimizations like KV-cache aware routing and speculative decoding, to achieve production-scale inference for large models.