Coreweave

May 20, 2026
SummaryAI, from the source post

This post details the performance gains achieved by CoreWeave's new NVIDIA GB300 NVL72 instances, featuring NVIDIA Blackwell Ultra GPUs, when running the DeepSeek R1 reasoning model. It quantifies a 6.5x performance improvement on inference compared to a 16-GPU NVIDIA H100 system, attributing this to the GB300's superior memory and interconnect bandwidth, which allows for 4-way Tensor Parallelism (TP4) versus TP16 on H100, significantly reducing communication overhead. The post also elaborates on the GB300 NVL72 architecture, including its 37TB memory, 130TB/s NVLink bandwidth, and integration with NVIDIA Quantum-X800 InfiniBand. CoreWeave's platform-specific optimizations, such as the Rack LifeCycle Controller, topology-aware scheduler leveraging NVLink labels, and observability through Grafana dashboards, are presented as key enablers of this record performance.

More from the ML Inference Benchmarking trail