Inside NVIDIA Rubin GPU Architecture: Powering the Era of Agentic AI | NVIDIA Technical Blog

This post introduces the NVIDIA Rubin GPU architecture and the Vera Rubin platform, detailing hardware innovations that accelerate agentic AI workloads. Key contributions include: 1. Rubin GPU architecture: 336 billion transistors, 224 SMs, 896 Tensor Cores with expanded precision, third-generation Transformer Engine (up to 50 petaflops NVFP4), 288 GB HBM4 memory (22 TB/s bandwidth), NVLink 6, NVLink-C2C, PCIe Gen 6, and Confidential Computing. 2. MoE acceleration: Inline descriptor updates for Tensor Memory Accelerator (TMA) simplify descriptor sharing and reduce data-movement overhead for Mixture-of-Experts models. 3. K-dimension optimization: Doubling the K-dimension processing per clock reduces loop iterations and improves Tensor Core utilization for both context and decode GEMMs at high tensor-parallel scale. 4. Rack-scale integration: The Vera Rubin NVL72 features liquid cooling, DSX MaxLPS for power management, cable-free MGX architecture, and hot-swappable NVLink switch trays for dense, resilient AI supercomputing.