Inside NVIDIA Rubin GPU Architecture: Powering the Era of Agentic AI | NVIDIA Technical Blog

What this post added

This post introduces the NVIDIA Rubin GPU architecture and the Vera Rubin platform, detailing hardware advancements specifically designed to accelerate agentic AI workloads. Key contributions include: 1. Rubin GPU architecture: 336 billion transistors, 224 SMs, 896 Tensor Cores with expanded precision, third-generation Transformer Engine (up to 50 petaflops NVFP4), 288 GB HBM4 (22 TB/s bandwidth), NVLink 6 (3,600 GB/s), PCIe Gen 6, and Confidential Computing with TEE-I/O. 2. MoE optimization: Enhanced Tensor Memory Accelerator with inline descriptor updates for TMA, reducing metadata-management and data-movement overhead for MoE models. 3. GEMM acceleration: Doubled Tensor Core throughput along the K-dimension, enabling fewer iterations for GEMMs and improving efficiency for both context and decode operations at high tensor-parallel scale. 4. Vera Rubin NVL72 platform: Integration of liquid cooling, DSX MaxLPS power smoothing, cable-free MGX architecture, and hot-swappable NVLink switch trays for rack-scale deployment of multitrillion-parameter models.