NVIDIA H100 Benchmarks for Large-Scale Training | CoreWeave

This post details a benchmarking study on NVIDIA H100 GPUs for large-scale training, focusing on reliability and performance. It highlights the impact of infrastructure architecture, including bare-metal hardware (dual-fabric InfiniBand/Ethernet), topology-aware scheduling (SUNK), and optimized storage/checkpointing (Tensorizer, custom tokenizer), on achieving higher MTTF (3.66 days at 1024 GPUs, a 10x improvement) and MFU (51-52% vs. 35-45% typical). The study also quantifies the business impact of these improvements, showing faster model delivery and reduced iteration time.