NVIDIA Blackwell Tops MLPerf Training 6.0 with Industry-Leading Scale and Performance | NVIDIA Technical Blog

What this post added

This post details the use of NVIDIA's scale-out networking platforms, specifically Spectrum-X Ethernet and Quantum InfiniBand, to achieve unprecedented scale and throughput in MLPerf Training 6.0. It highlights advanced adaptive routing and congestion control mechanisms within Spectrum-X to manage low-entropy, bursty traffic from MoE models, ensuring effective bandwidth near theoretical capacity and balancing tail latency. The post also showcases the integration of these networking capabilities with software optimizations like full-iteration CUDA graphs, CuTe DSL kernel fusions, MXFP8 attention blocks, and router/hybrid EP optimizations to achieve record-breaking training times for large MoE models across up to 8,192 Blackwell GPUs.