NVIDIA Blackwell Tops MLPerf Training 6.0 with Industry-Leading Scale and Performance | NVIDIA Technical Blog

This post details NVIDIA's leading results in MLPerf Training v6.0, showcasing the performance of the Blackwell platform across various AI models, including large MoE models like DeepSeek-V3 and GPT-OSS-20B. It highlights the use of 8,192 Blackwell GPUs in production cloud environments and the critical role of software innovations such as full-iteration CUDA graphs for token-dropless MoEs, CuTe DSL-enabled kernel fusions, MXFP8 attention block, and advanced MoE router and hybrid EP optimizations. The post also emphasizes the contribution of Spectrum-X Ethernet's Advanced Adaptive Routing and Congestion Control for efficient scale-out networking, ensuring sustained effective bandwidth and balanced tail latency. Specific optimizations like full-iteration CUDA graphs with paged stashing and synchronization-free operators, CuTe DSL for advanced kernel fusions, MXFP8 attention for improved precision, fused router kernels, and 1F1B all-to-all overlap optimizations are detailed, contributing to significant end-to-end speedups.