CoreWeave Leads on Artificial Analysis for Kimi K2.7 Code Inference | CoreWeave Blog

This post details the engineering work involved in optimizing Kimi K2.7 Code for production inference on CoreWeave's infrastructure. Key contributions include: - Metal-to-model optimization: Applying NVFP4 quantization to leverage Blackwell's FP4 execution and integrating a DFlash speculative decoder. - Quantization process: Using NVIDIA's Model-Optimizer with a custom 3-pass calibration on general conversation, math, and coding datasets. - Speculative decoding training: Generating a new dataset using the NVFP4 checkpoint and training a DFlash speculative decoding model with the D-PACE loss function. - Benchmarking: Utilizing AIPerf from the NVIDIA Dynamo team and contributing support for common AL evals to measure acceptance length and compare results. - Infrastructure integration: Training occurred on GB300 NVL72 racks managed by CoreWeave Kubernetes Service over a RoCE fabric. Production endpoint uses NVFP4 + DFlash on vLLM by default. - Performance validation: Achieving leading price-performance as validated by Artificial Analysis.