TechPrufer
Get started
NVIDIA
NVIDIA logoNVIDIA · Feature trail

NVLink Scale-Up Networking for AI Factories

10 posts · May 2026 – Sep 2026

What this trail covers

This feature thread tracks the evolution of NVIDIA NVLink and related networking technologies for AI factories, focusing on enabling high-bandwidth, low-latency GPU-to-GPU communication. The Sixth Generation NVLink interconnect with NVLink 6 Switch provides the highest GPU-to-GPU bandwidth at the lowest latency all-to-all topology for scale-up networking, as well as support for SHARP in-network compute for offloading collective operations. It also includes rack-level resiliency features designed to support agentic AI workloads by providing predictable, low-latency, and high-throughput communication across thousands of accelerators. The NVIDIA Vera Rubin Platform, with its LPU C2C technology, extends this deterministic execution model across many LPUs, enabling multi-trillion parameter models at scale by combining high-radix point-to-point links, compiler-scheduled data movement, and hardware-driven plesiosynchronous timing to achieve rack-scale determinism and unified on-chip SRAM.

All NVIDIA trails
  1. How NVIDIA NVLink 6 Delivers Multi-Layer Resiliency for AI Factories | NVIDIA Technical BlogThis post details the multi-layer resiliency features of NVIDIA NVLink 6 for AI factories. It explains how NVLink 6 achieves a natively lossless fabric through physical layer error correction (lightweight FEC, PLR, UPHY recovery) and link layer credit-based flow control (CBFC) to eliminate packet loss. It also highlights system-level resilience with redundant NMX Controllers and dual management paths, and application-level resilience through features like Shadow Engine Recovery and NCCL Elasticity Support. The post emphasizes how these integrated layers ensure continuous operation and maximum cluster productivity by detecting, containing, and recovering from transient errors and link degradations.
  2. NVIDIA NVLink Fusion Brings NVHBM to Next-Generation AI Infrastructure | NVIDIA Technical BlogThis post introduces NVIDIA NVLink Fusion, a connective technology and IP that enables hyperscalers and AI-native companies to deploy custom XPUs and CPUs into the NVIDIA AI infrastructure platform using the MGX rack-scale architecture. It details NVHBM, a custom HBM base-die technology that offers up to 30% more memory bandwidth, up to 67% reduction in PHY and support area (leading to up to 30% more main-die silicon for compute), and up to 15% lower HBM power usage compared to standard HBM4e. The combination of NVLink Fusion and NVHBM is shown to deliver a 30% overall end-to-end performance increase per XPU at rack scale by compounding bandwidth, area, and power improvements.
  3. Giga-Scale AI and the Ethernet Evolution: How Spectrum-X Ethernet Rewrites the Rules | NVIDIA Technical BlogThis post introduces Spectrum-X Ethernet as a solution to the limitations of traditional Ethernet for giga-scale AI workloads. It details the three hardware-accelerated control loops (In-switch Adaptive Routing, Targeted Congestion Control, NIC-based Plane Load Balancing) that enable predictable low latency and high fabric utilization. It also explains the Multiplane technology, which decomposes host bandwidth into independent network planes to scale fabrics efficiently and resiliently, demonstrating superior performance and isolation compared to traditional Ethernet, especially under failure conditions and multi-tenant noise traffic.
  4. How NVIDIA Groq 3 LPX Unlocks Ultrafast Interactivity at Long Context on NVIDIA Vera Rubin | NVIDIA Technical BlogThis post details how NVIDIA Groq 3 LPX, when paired with Vera Rubin NVL72, achieves ultrafast interactivity at long context for agentic AI sessions. It highlights the deterministic execution model of Groq 3 LPX, which uses compiler-scheduled chip-to-chip communication and compute-communication overlap to minimize first-bit latency. The post explains how this approach addresses the challenges of serving models at high token rates with large KV caches, contrasting it with traditional tensor parallelism limitations at small batch sizes. Performance benchmarks on Gemma 4 31B at 100K context are presented, demonstrating median speeds of 3,431 output tokens/second.
  5. Setting a World Record for MoE Pre-Training on NVIDIA GB300 NVL72 | NVIDIA Technical BlogThis post details the world-record pre-training of DeepSeek-V3 671B on NVIDIA GB300 NVL72, achieving 1,648 TFLOPs per GPU. It highlights the critical role of tightly coupled scale-up (NVLink) and scale-out (ConnectX-8 SuperNICs, Quantum-X800 InfiniBand/Spectrum-X Ethernet) networking for MoE architectures. The post emphasizes the co-design of hardware (GB300 NVL72, NVLink 6, SuperNICs) and software (Megatron Core, TorchTitan, JAX) to maximize delivered FLOPs, showcasing a 3x performance increase over previous generations and a 1.5x improvement through software optimization alone on the same hardware.
  7. Inside NVIDIA Rubin GPU Architecture: Powering the Era of Agentic AI | NVIDIA Technical BlogThis post introduces the NVIDIA Rubin GPU and Vera platform, detailing architectural enhancements for agentic AI workloads. Key contributions include: 1. Enhanced NVLink 6 providing 3,600 GB/s scale-up bandwidth for all-to-all GPU-GPU communication, and NVLink-C2C for 1,800 GB/s coherent CPU-GPU communication, crucial for scaling multitrillion-parameter models. 2. Improvements in MoE weight and token movement through inline descriptor updates for TMA, reducing metadata-management and data-movement overhead. 3. Doubling of Tensor Core throughput per clock by processing a larger K-dimension, reducing loop overhead and improving efficiency for context and decode GEMMs at high tensor-parallel scale. 4. Introduction of the Vera Rubin NVL72 rack with liquid cooling, power smoothing, cable-free MGX architecture, and hot-swappable NVLink switch trays to enable higher GPU density and throughput.
  8. NVIDIA NVLink: The Scale-Up Network for AI Factories | NVIDIA Technical BlogThis post details the Sixth Generation NVLink interconnect with NVLink 6 Switch, highlighting its role as a purpose-built scale-up networking fabric for AI factories. It emphasizes the high GPU-to-GPU bandwidth (up to 3.6 TB/s per GPU), low latency, all-to-all topology, and support for SHARP in-network compute. The post also discusses rack-level resiliency features, extreme co-design across the stack, and its integration with software like NVIDIA Dynamo and TensorRT-LLM. Performance comparisons show up to 2.3X higher decode throughput compared to off-the-shelf Ethernet for MoE and LLM workloads. Key metrics for evaluation are presented: Delivered Performance, Factory Resiliency, and Platform Maturity and Proven Supply Chain.
  9. Enhancing Goodput in Large-Scale LLM Training with Nonuniform Tensor Parallelism | NVIDIA Technical BlogIntroduces Nonuniform Tensor Parallelism (NTP) as an experimental framework to enhance Goodput in large-scale LLM training. NTP dynamically adjusts the tensor parallelism degree in response to transient GPU unavailability, preventing training stalls and throughput loss. It also proposes dynamic power boosting to compensate for performance loss in affected scale-up domains and employs efficient, overlapped tensor resharding techniques that minimize overhead to less than 1%. This enables resilient operation even as scale-up domains grow to 72 GPUs on NVIDIA Blackwell and Blackwell Ultra systems.
  10. Maximize AI Factory Energy Efficiency Through Full-Stack Inference and Training Optimizations | NVIDIA Technical BlogThis post details how NVIDIA maximizes AI factory energy efficiency through full-stack inference and training optimizations. It highlights system co-design with power, cooling, and infrastructure, and collaboration with ecosystem partners. Key technical contributions include the use of the NVIDIA GB200 NVL72 rack-scale system with direct-to-chip liquid cooling and in-rack power smoothing, NVIDIA DSX for dynamic power allocation and real-time telemetry, and the adoption of narrow precision formats like NVFP4 for improved throughput and energy efficiency. For LLM training, it discusses energy-aware techniques such as coordinated GPU speed tuning to minimize idle time and reduce energy consumption without increasing training time, and fine-grained profiling of kernel and phase-level energy usage in collaboration with the ML.ENERGY Initiative and Megatron-LM.
  11. NVIDIA Blackwell Tops MLPerf Training 6.0 with Industry-Leading Scale and Performance | NVIDIA Technical BlogThis post details NVIDIA's leading results in MLPerf Training v6.0, showcasing the performance of the Blackwell platform across various AI models, including large MoE models like DeepSeek-V3 and GPT-OSS-20B. It highlights the use of 8,192 Blackwell GPUs in production cloud environments and the critical role of software innovations such as full-iteration CUDA graphs for token-dropless MoEs, CuTe DSL-enabled kernel fusions, MXFP8 attention block, and advanced MoE router and hybrid EP optimizations. The post also emphasizes the contribution of Spectrum-X Ethernet's Advanced Adaptive Routing and Congestion Control for efficient scale-out networking, ensuring sustained effective bandwidth and balanced tail latency. Specific optimizations like full-iteration CUDA graphs with paged stashing and synchronization-free operators, CuTe DSL for advanced kernel fusions, MXFP8 attention for improved precision, fused router kernels, and 1F1B all-to-all overlap optimizations are detailed, contributing to significant end-to-end speedups.
  12. Unlock Exascale Performance on NVIDIA GB200 NVL72 with Slurm Topology-Aware Job Scheduling | NVIDIA Technical BlogThis post details the integration of Slurm's new topology/block plugin (version 23.11) with the NVIDIA GB200 NVL72 system to enable topology-aware job scheduling. It explains how this plugin aligns jobs with NVL72 domain boundaries, minimizing fragmentation and optimizing GPU occupancy. The post also discusses how GB200 NVL72 supports larger job segment sizes (up to 18 nodes) compared to previous systems, benefiting high I/O workloads like MoE training, and provides best practices for segment sizing based on simulation results for maximizing cluster utilization.
  13. How the NVIDIA Vera Rubin Platform is Solving Agentic AI’s Scale-Up Problem | NVIDIA Technical BlogThis post details how the NVIDIA Vera Rubin Platform, leveraging LPU C2C technology, addresses the scale-up problem for agentic AI workloads. It introduces the concept of extreme co-design where networking fabric is designed with silicon, compiler, and serving stack to achieve predictable, low-latency, and high-throughput communication across thousands of accelerators. Key technical contributions include: high-radix point-to-point links (96 C2C links at 112 Gbps per LPU), compiler-scheduled data movement (statically planned transfers of 320-byte vectors), and hardware-driven plesiosynchronous timing to cancel clock drift and align LPUs. This enables rack-scale determinism with a unified 128 GB on-chip SRAM pool, allowing trillion-parameter models to run at low latency without compromising context window or accuracy, and bounding tail latency under bursty fan-out patterns.