CoreWeave Delivers Breakthrough AI Performance with NVIDIA GB200 and H200 GPUs in MLPerf Inference v5.0

This post details CoreWeave's MLPerf Inference v5.0 submission results, highlighting breakthrough performance with NVIDIA GB200 Grace Blackwell Superchips and H200 GPUs. It quantifies performance gains: 800 TPS on Llama 3.1 405B with GB200 (2.86X per-chip speedup over H200) and 33,000 TPS on Llama 2 70B with H200 (40% improvement over H100). The post explains how CoreWeave Cloud Platform optimizations, including bare-metal infrastructure, advanced networking, Mission Control for resource management, CKS for inference engines, SUNK for efficient scheduling, and Tensorizer for model loading, contribute to these results. It also provides specific benchmark numbers and comparisons against previous generations and competitors.