CoreWeave Drives 20% Higher GPU Cluster Performance | CoreWeave

This post quantifies CoreWeave's AI infrastructure efficiency by demonstrating up to 20% higher Model FLOPS Utilization (MFU) compared to public benchmarks for Llama 3.1 training runs on 128 NVIDIA H100 GPUs. It shows MFU of 51.9% (a 28% increase over Aleph Alpha's A100 results) and 49.2% (an 18% increase over MosaicML's H100 results). The post attributes these gains to optimizations across the stack, including bare-metal Kubernetes, high-performance networking (InfiniBand and Spectrum switches), optimized asynchronous checkpointing (Tensorizer), and high-speed object storage (2 GB/s per GPU). It also mentions parity with NVIDIA's DGX Cloud Benchmarking Recipes.