TechPrufer
Get started
Blogs
NVIDIA logo

NVIDIA

Engineering blog · developer.nvidia.com/blog

201posts10this week0watchers

WatchOpen blog (opens in a new tab)
PostsFeature trails66Interview Qs83Salaries

Feature trails

66
Feature trailActive

AI-Assisted Scientific Simulation Workflow Generation

This feature thread tracks the evolution of AI-assisted tools and workflows for scientific simulation, focusing on enabling researchers to generate complex simulation code and pipelines using natural language prompts. Initial efforts focused on the NVIDIA ALCHEMI Toolkit, providing composable, PyTorch-native, GPU-accelerated building blocks for Machine Learning Interatomic Potentials (MLIP). Subsequent developments have integrated AI coding agents with ALCHEMI Toolkit agent skills and reference materials to automate the conversion of CAD assets into simulation-ready OpenUSD assets for robotics. This includes using AI models to guide the import of STEP files, match visual appearance, configure physics properties, validate against SimReady requirements, and run pick-and-place tasks in Isaac Sim. Frontier AI models like GPT-6 Astra can assist by writing Python code that calls Omniverse tools at each stage, reducing manual preparation effort. Physics configuration uses manufacturer datasheets and URDFs to define joint axes, limits, and mass properties, with contact friction coefficients. SimReady validation checks units, geometry, materials, rigid bodies, joints, drives, articulation, and runtime behavior.

3 posts · 2026

Timeline

Feature trailActive

Agentic AI Infrastructure Acceleration with BlueField DPUs

This feature thread tracks the evolution of infrastructure designed to support agentic AI workloads, particularly focusing on the challenges and advancements in large-scale model pre-training and deployment. Initial efforts focused on agentic AI introducing complex distributed workflows requiring tight coupling of GPUs, CPUs, memory, networking, storage, and security. This demanded infrastructure capable of moving, protecting, retrieving, and reusing context data at high speeds without latency. Subsequent developments have focused on building reliable data analytics agents, emphasizing constrained harnesses, unified data access, schema context seeding, opinionated toolsets, and effective handling of structured and unstructured data sources like documents and videos. Techniques include preprocessing data, limiting the agent's action space, and making every attempt inspectable through execution traces and specialized agents for failure analysis. This extends to optimizing the interaction between agents and underlying infrastructure by focusing on the harness as a key optimization surface, especially when working with smaller, fixed LLMs.

72 posts · 2026

Timeline

Feature trailActive

Full-Stack Functional Safety System for Robotics

This feature thread tracks the evolution of full-stack functional safety systems for robotics, building upon NVIDIA's extensive experience in autonomous vehicle safety. It encompasses the integration of specialized hardware like NVIDIA IGX Thor with a comprehensive safety OS (Halos OS) to provide standards-compliant, extensible safety solutions for industrial robots, humanoids, and AMRs. Key advancements include the development of safety islands, high diagnostic coverage, and mechanisms for free-wheeling operation. The latest developments focus on enabling robots to perform complex assembly tasks, such as busbar assembly and multi-connector insertion, by combining classical robotics techniques with advanced AI and simulation. This includes the use of FoundationPose for perception, Lissajous-curve manipulation primitives for planning, and impedance controllers for precise execution. For more complex tasks like multi-connector insertion, new frameworks like Deep Object Pose Estimation Revisited (DOPER) are developed, and sim-to-real reinforcement learning is employed for policy training. Reusable, containerized components and orchestration systems like TALOS accelerate development and deployment.

3 posts · 2026

Timeline

Feature trailActive

GPU-Accelerated Scientific Image Analysis Pipeline

This feature thread tracks the evolution of GPU-accelerated pipelines for scientific image analysis, focusing on reducing data loading, alignment, and processing times for high-throughput instruments. Initial efforts focused on addressing the significant data movement and processing bottlenecks in traditional CPU-bound pipelines for domains like astronomy and X-ray science. NVIDIA cuPhoton provides a suite of GPU-native building blocks that keep image data on the GPU from sensor read-through classification, collapsing processing times from months to minutes. This enables interactive analysis and allows facilities to keep pace with data generation rates. The toolkit scales across multi-GPU, multi-node systems, demonstrating significant speedups in image loading and signal processing.

1 post · 2026

Timeline

Feature trailActive

AI Infrastructure Performance Tuning

This feature thread tracks the evolution of optimizing AI infrastructure performance, focusing on identifying and resolving configuration gaps that impact training throughput. Initial efforts focused on systematic debugging of common performance bottlenecks across hardware, software, and networking layers. Subsequent developments detail advanced diagnostic techniques, best practices for system configuration, and the impact of these optimizations on large-scale AI deployments, including specific optimizations for bitwise-deterministic pretraining with NVIDIA Megatron Core, reducing the determinism tax to approximately 2% at 2,432 GPUs while maintaining bitwise determinism over 800 steps.

4 posts · 2026

Timeline

Feature trailActive

GPU-Initiated Communication for Molecular Dynamics

This feature thread tracks the evolution of GPU-initiated communication for molecular dynamics simulations at scale. Initial efforts focused on addressing the scalability bottlenecks of traditional MPI-based halo exchange, which relied heavily on CPU orchestration and led to significant synchronization waits. Subsequent developments have introduced GPU-native communication using NVSHMEM, enabling device-side packing, transfer, and signaling within fused kernels. This approach eliminates CPU-GPU communication overhead. DOCA GPUNetIO further unifies this by providing a common GDA-KI foundation across various communication libraries like NCCL and NVSHMEM, allowing CUDA kernels to directly drive Ethernet, RDMA, Verbs, and DMA operations, removing the CPU from the application critical path and enabling lower latency and higher efficiency for distributed applications.

2 posts · 2026

Timeline

Feature trailActive

GPU Cluster Configuration Management

This feature thread tracks the evolution of managing GPU-accelerated Kubernetes clusters, focusing on ensuring compatibility and verifiability across numerous interdependent components. Initial efforts addressed the challenges of version conflicts and the difficulty of tracing errors in complex, multi-component environments. NVIDIA AI Cluster Runtime (AICR) was developed to provide version-locked, validated recipes that define compatible component combinations. The v1.0 release establishes a stable compatibility contract for its CLI, API, SDK, and artifact schemas, enabling operators and integrators to build with confidence. AICR's four core capabilities—Snapshot, Recipe, Bundle, and Validation—allow for recording observed state, describing desired configurations, rendering deployment artifacts, and verifying running clusters against recipes with signed evidence. Ecosystem integrations with tools like Pulumi and Mirantis's k0rdent demonstrate the consumption of these configurations through various infrastructure-as-code and multi-cluster management platforms. Future developments will likely focus on expanding recipe coverage, enhancing validation capabilities, and further integrating with the broader cloud-native ecosystem.

1 post · 2026

Timeline

Feature trailActive

GPU Resource Partitioning with Green Contexts

This feature thread tracks the evolution of GPU resource management and partitioning within a single process. Initial efforts focused on addressing the challenges of concurrent workloads interfering with each other on shared GPU resources, leading to unpredictable performance and increased latency. The introduction of CUDA Green Contexts in CUDA 12.4 (Driver API) and CUDA 13.1 (Runtime API) provides explicit control over GPU execution resources like SMs and workqueues. This allows applications to partition dedicated subsets of SMs and provision workqueues independently, enabling latency-sensitive kernels to run without being blocked by bulk workloads. This significantly reduces critical kernel latency and improves overall GPU utilization by isolating competing tasks.

1 post · 2026

Timeline

Feature trailActive

Multi-Device AI Inference with TensorRT

This feature thread tracks the evolution of enabling high-performance AI inference across multiple GPUs and edge devices. Initial efforts focused on the challenges of scaling generative AI workloads beyond single-GPU memory and compute limits. TensorRT 11.0 introduced native multi-device inference support, leveraging NVIDIA NCCL for distributed collectives and enabling seamless scaling of generative AI pipelines. This includes the implementation of context parallelism strategies like AllGather K. The DIN Deploy samples extend this by providing C++ implementations for local AI applications using ONNX Runtime and TensorRT RTX, demonstrating performance gains for various models and enabling graphics interop for advanced use cases.

5 posts · 2026

Timeline

Feature trailActive

Automated Model Fine-Tuning and Deployment with TAO Agent Skills

This feature thread tracks the advancement of automated model fine-tuning and deployment, focusing on streamlining the process for NVIDIA's foundation models. Initial efforts likely involved manual fine-tuning and deployment pipelines. Subsequent developments have introduced tools and frameworks like NVIDIA TAO and agent skills to automate these processes, significantly reducing engineering overhead and time-to-deployment. This includes advancements in techniques like LoRA for efficient adaptation, and now extends to fine-tuning large language models like Nemotron for specific dialects and languages, incorporating techniques like minimal curation, weighted replay mixing, and partial encoder unfreezing to adapt models while retaining existing language capabilities. The workflow is being extended to include diarization and is being made reproducible via notebooks and agent skills.

13 posts · 2026

Timeline

Feature trailActive

Generative Recommender Systems

This feature thread tracks the evolution of Generative Recommender Systems (GRs), a paradigm shift from traditional embedding-based approaches. Initial efforts focused on architectural shifts towards sequence modeling objectives and transformer-like architectures, exemplified by models like Hierarchical Sequential Transduction Units (HSTU) and Semantic IDs, which address challenges in scalability, cold start, and the long-tail problem. The NVIDIA recsys-examples repository provides production-ready examples. Subsequent developments have integrated HSTUs with NVIDIA Dynamo-Triton for low-latency inference, leveraging PyTorch Ahead-of-Time (AOTI) compilation for native C++ artifacts, FlexKV-backed KV caching to avoid recomputing long user histories, and NV embedding cache for efficient memory usage. This enables significant performance improvements, with up to 5.93x lower latency demonstrated for an eight-layer HSTU model at dynamic batch size 8 with a 100% GPU KV-cache hit rate.

2 posts · 2026

Timeline

Feature trailActive

Accelerated AI Storage Access with RDMA

This feature thread tracks the evolution of accelerated storage access for AI workloads, focusing on enabling direct, RDMA-based data transfers between compute accelerators (GPUs, TPUs, XPUs) and high-capacity file and object storage. Initial efforts focused on addressing the need for zero-copy, CPU-offloaded data transfers to support the increasing demands of AI training, fine-tuning, and inference. The development of standardized APIs and wire protocols, such as NVIDIA cuObject and the SCADA Server SDK, aims to improve interoperability across storage providers and cloud services. The xio-sig consortium plays a key role in defining these open industry standards. Subsequent developments will likely focus on broader adoption, performance optimizations, and integration with emerging storage technologies.

1 post · 2026

Timeline

Feature trailActive

Secure AI Agent Deployment

This feature thread tracks the evolution of secure AI agent deployment, focusing on architectural controls to mitigate risks associated with connecting AI agents to live tools and corporate data. Initial efforts focused on identifying common failure modes such as inadequate access control, arbitrary code execution via agent tools, lack of network egress controls, and exposure of plaintext secrets. Subsequent developments have introduced deterministic controls like robust agent access controls, credential management, and formal policy analysis via NVIDIA OpenShell, enabling sandboxed execution and controlled service access without rewriting agents. Organizations like Cadence, Slack, and Gecko Robotics are adopting OpenShell for chip design, enterprise automation, and physical robotics governance.

3 posts · 2026

Timeline

Feature trailActive

AI Factory Power Management and Optimization

This feature thread tracks the evolution of AI factory infrastructure, focusing on maximizing computational output within fixed power and thermal constraints. Initial efforts focused on understanding the power demands of AI workloads and the inefficiencies of static rack provisioning. NVIDIA DSX MaxLPS has been introduced as a suite of technologies to address these challenges. Key components include Dynamic Power Software (DPS) for real-time monitoring and reallocation of power headroom, workload variability into managed capacity, and policy-governed power sharing to dynamically allocate power across participating resources, enabling customers to deploy up to 40% more GPUs within the same approved power budget. Evaluations have shown significant increases in aggregate throughput and throughput per provisioned watt, while maintaining per-instance performance and acceptable tail latency. A repeatable validation method for operators has been established.

3 posts · 2026

Timeline

Feature trailActive

Dense vs. Mixture-of-Experts (MoE) Model Architectures

This feature thread tracks the evolution of understanding and deploying different AI model architectures, specifically comparing Dense models with Mixture-of-Experts (MoE) models. Initial efforts focused on the fundamental differences in parameter activation and their impact on performance. This post details the introduction of advanced fused MLP kernels for MoE models, custom-built with the NVIDIA CuTe DSL. These kernels address memory and synchronization bottlenecks, enabling sync-free MoE execution. Subsequent developments have introduced TE's GroupedLinear for reduced kernel launch overhead, MXFP8 precision for memory reduction, and a fused GroupedMLP kernel that combines quantization, SwiGLU activation, and routing-weight scaling, significantly improving training throughput for MoE-based biological foundation models.

2 posts · 2026

Timeline

Feature trailActive

Tenant Isolation for Kubernetes on Shared GPU Infrastructure

This feature thread tracks the evolution of enabling isolated tenant Kubernetes clusters on shared GPU infrastructure. Initial efforts focused on addressing the challenges of managing multiple teams with conflicting requirements (CRDs, RBAC, Kubeflow versions) on shared hardware. The solution combines KAI Scheduler for topology-aware, hierarchical GPU scheduling with per-team quotas and dynamic allocation, and vCluster for provisioning fully isolated tenant clusters with independent control planes. Subsequent developments have introduced NodeWright for declarative, workload-aware host operating system management, enabling safe, progressive rollouts of system-level changes (kernel tuning, CVE remediation, security agent installation) across fleets without disrupting running workloads. NodeWright integrates with NVIDIA AI Cluster Runtime and DSX OS, providing a robust mechanism for managing the underlying infrastructure of shared GPU clusters.

4 posts · 2026

Timeline

Feature trailActive

Hardware-Rooted AI Security with Confidential Computing

This feature thread tracks the evolution of hardware-rooted AI security, focusing on NVIDIA Confidential Computing (CC) to protect data, code, and model integrity during inference with minimal performance overhead. Initial efforts focused on the challenges of securing AI workloads, leading to the development of CC integrated into Blackwell GPUs. Subsequent developments have demonstrated near-native inference performance (up to 98%) through optimizations in secure work submission, bandwidth management, and framework adaptations in TensorRT LLM for host-to-device data movement, kernel autotuner timing stabilization, and CC-aware multi-GPU communication. This enables running production AI inference on trusted hardware with minimal performance impact.

2 posts · 2026

Timeline

Feature trailActive

Slurm Topology-Aware Job Scheduling for GB200 NVL72

This feature thread tracks the evolution of job scheduling for large-scale AI and HPC clusters, focusing on optimizing resource utilization and performance by understanding the underlying hardware topology. Initial efforts focused on traditional scheduling approaches. Subsequent developments have introduced topology-aware scheduling plugins for workload managers like Slurm, specifically designed for modern architectures like NVIDIA GB200 NVL72. This includes the development of topology/block plugins and tools like NVIDIA Topograph to discover and publish cluster topology information to schedulers, enabling efficient workload placement within locality domains and across systems and racks. This ensures GPUs are placed in the most efficient locations, minimizing contention and latency, and maximizing throughput and resource utilization.

2 posts · 2026

Timeline

Feature trailActive

Guided Generative Models for Rare Event Likelihood Estimation

This feature thread tracks the development and application of guided generative models, particularly diffusion models, for efficiently estimating the likelihood of rare, high-impact events. Initial work focused on climate science, using models like NVIDIA cBottle and tools like Earth2Studio, to steer generative processes towards extreme weather phenomena (e.g., tropical cyclones) and then employing odds-ratio diagnostics to correct for sampling bias. Subsequent developments have demonstrated the effectiveness of Score-Based Data Assimilation (SDA) for regional models, constraining diffusion models with point observations to improve forecasts and reduce error metrics like wind-speed RMSE. HealDA has been introduced for global atmospheric state estimation in seconds by mapping heterogeneous remote-sensing and in situ observations onto a HEALPix grid. Earth2Studio data sources provide unified access to various observation types for developing and validating weather models.

2 posts · 2026

Timeline

Feature trailActive

LLM Serving Simulation and Optimization

This feature thread tracks the evolution of tools and techniques for simulating and optimizing Large Language Model (LLM) serving stacks. Initial efforts focused on the complexity of tuning LLM deployments due to interacting choices in model backends, tensor parallelism, scheduling, routing, and caching. DynoSim has been introduced as a workload-driven discrete-event simulation of the NVIDIA Dynamo serving stack, combining measured engine timings, scheduler behavior, router and planner logic, KV cache performance, and GPU telemetry. AIPerf is a new multiprocess benchmarking tool that replaces GenAI-Perf, offering more accurate server benchmarking by preventing client-side bottlenecks. It supports a wide range of endpoint types, public datasets, and configurable arrival patterns for realistic workload testing. Core metrics like TTFT, ITL, request latency, and output token throughput are reported with percentile breakdowns and GPU telemetry.

2 posts · 2026

Timeline

Feature trailActive

Local Network Inference Routing

This feature thread tracks the evolution of local network inference routing, enabling distributed AI workloads across multiple compatible systems. Initial efforts focused on addressing multi-agent bottlenecks by introducing NVIDIA PAIR, a virtual inference router that distributes independent inference requests to available nodes on a local network. PAIR supports familiar interfaces like Ollama and LM Studio without requiring agent harness changes. It leverages mDNS for discovery, MTLS for encryption, and now TensorRT Edge-LLM on Jetson AGX Thor for accelerated agentic AI inference, achieving significant performance gains in benchmarks like MLPerf Edge Agentic through techniques such as NVFP4 quantization, KV cache reuse, and tree-based multi-token prediction.

4 posts · 2026

Timeline

Feature trailActive

CUDA Toolkit Enhancements

This feature thread tracks the evolution of the CUDA Toolkit, encompassing new platform support, architectural previews, enhanced GPU management, and expanded CUDA Python and CCCL functionality. CUDA Toolkit 13.3 introduces support for Windows on Arm, a preview of the NVIDIA Rubin architecture, modernized Multi-Process Service (MPS) V3 for precise GPU partitioning, and CUDA Compute Fabric Transport (CFT) for advanced inter-GPU communication. It also enhances CUDA Python with texture/surface programming and introduces the `clmad` instruction for hardware-accelerated binary field arithmetic. The thread also covers agentic AI skills for translating CUDA tile operations between Python and Rust, demonstrating a new paradigm for developer tooling and code migration.

2 posts · 2026

Timeline

Feature trailActive

NVLink Scale-Up Networking for AI Factories

This feature thread tracks the evolution of NVIDIA NVLink and related networking technologies for AI factories, focusing on enabling high-bandwidth, low-latency GPU-to-GPU communication. The Sixth Generation NVLink interconnect with NVLink 6 Switch provides the highest GPU-to-GPU bandwidth at the lowest latency all-to-all topology for scale-up networking, as well as support for SHARP in-network compute for offloading collective operations. It also includes rack-level resiliency features designed to support agentic AI workloads by providing predictable, low-latency, and high-throughput communication across thousands of accelerators. The NVIDIA Vera Rubin Platform, with its LPU C2C technology, extends this deterministic execution model across many LPUs, enabling multi-trillion parameter models at scale by combining high-radix point-to-point links, compiler-scheduled data movement, and hardware-driven plesiosynchronous timing to achieve rack-scale determinism and unified on-chip SRAM.

10 posts · 2026

Timeline

Feature trailActive

Federated Learning for Multimodal AI

This feature thread tracks the evolution of federated learning for multimodal AI, focusing on enabling collaborative training of vision-language models (VLMs) and unified multimodal models (UMMs) across distributed data silos. Initial efforts focused on the challenges of orchestrating federated training for VLMs, particularly the large size of model updates and the heterogeneity of data and task mixes across sites. NVIDIA FLARE has been developed to address these challenges by supporting parameter-efficient fine-tuning and automated research loops. Recent advancements introduce AI agents and Auto-FL to automate the research process, enabling faster iteration on FL strategies, ensuring reproducibility through experiment ledgers, and incorporating literature-grounded recovery mechanisms to escape local optima. This evolution aims to accelerate the evaluation of new FL algorithms and model architectures.

2 posts · 2026

Timeline

Feature trailActive

GPU Acceleration for Genomics and Protein Folding Workloads

This feature thread tracks the evolution of GPU acceleration for genomics and protein folding workloads, enabling faster and more accurate analysis. Initial efforts focused on the general benefits of GPU acceleration in these fields. Subsequent developments have introduced specialized hardware like the NVIDIA RTX PRO 4500 Blackwell Server Edition GPU and software platforms like NVIDIA BioNeMo and Parabricks. These advancements significantly reduce processing times for tasks such as sequence alignment and variant calling, and accelerate protein structure prediction. This post details the performance gains of the RTX PRO 4500 Blackwell GPU for tools like Minimap2, fq2bam, and DeepVariant, and for protein structure inference with Openfold3 and cuEquivariance, demonstrating significant speedups over previous generations and enabling faster clinical decision-making and drug discovery.

1 post · 2026

Timeline

Feature trailActive

Model Quantization for Inference Optimization

This feature thread tracks the evolution of model quantization techniques to optimize AI model inference performance. Initial efforts focused on the general concepts and benefits of quantization, followed by post-training quantization (PTQ) methods using tools like NVIDIA Model Optimizer. This post details the process of converting FP8-quantized checkpoints into high-performance TensorRT inference engines, including ONNX export, TensorRT engine building, and performance profiling, demonstrating significant speedups. Subsequent developments introduce NVFP4, a 4-bit mixed-precision training recipe for JAX and MaxText on NVIDIA Blackwell and Rubin platforms, achieving up to 1.73x speedup over FP8 baselines with negligible accuracy loss by leveraging subbyte precision and native hardware support.

9 posts · 2026

Timeline

Feature trailActive

AI-Assisted Supply Chain Optimization

This feature thread tracks the evolution of AI-assisted supply chain optimization, focusing on integrating human expertise with quantitative models to improve decision-making. Initial efforts involved building digital command centers on platforms like Palantir Foundry to unify disparate data sources and provide real-time visibility. Subsequent developments have focused on leveraging specialized AI models, like NVIDIA Nemotron, trained on historical allocation decisions and outcomes, to augment quantitative solvers (e.g., NVIDIA cuOpt) and codify institutional knowledge. The goal is to create a feedback loop where accepted, edited, and overridden AI recommendations continuously refine the system, leading to improved accuracy and efficiency in complex supply chain operations.

1 post · 2026

Timeline

Feature trailActive

Modern C++ Runtime for CUDA

This feature thread tracks the evolution of modern C++ abstractions for fundamental CUDA programming model concepts, aiming to make CUDA C++ development safer and more convenient. Initial efforts focused on providing idiomatic C++ APIs for stream management, memory allocation, and kernel launches, leveraging modern C++ features and explicit dependency management. Subsequent developments have introduced the CCCL runtime, offering a collection of headers like `<cuda/stream>`, `<cuda/buffer>`, and `<cuda/launch>`, providing owning types and non-owning refs for resource management and improved composability with existing code.

5 posts · 2026

Timeline

Feature trailActive

Federated Identity Propagation

This feature thread tracks the evolution of propagating user identity across federated Kubernetes and AI platforms. Initial efforts focused on the challenges of maintaining a unified user experience and security posture across distributed environments. The central identity gateway pattern has emerged as a key solution, enabling a single sign-on experience and consistent identity context for users interacting with multiple clusters and AI services. This pattern decouples session ownership from request enforcement, allowing regional gateways to validate user identity through a shared session store instead of managing independent OIDC flows. At NVIDIA, this approach reduced repeated login events by 55% across internal developer platforms spanning Kubernetes clusters in AWS and OCI while enabling unified platform shells and AI assistants with delegated user identity. The architecture uses standard OpenID Connect, a minimal /gateway/userinfo validation endpoint, stateless regional gateways, and a shared Redis-backed session store with explicit TTLs. Centralized session ownership coordinates token refresh and logout platform-wide, reduces upstream identity-provider load to scale with active users rather than user-tool combinations, and standardizes identity headers for downstream services. Security guardrails include mutual TLS or workload identity between gateways, stripping inbound identity headers before injecting trusted ones, and defining explicit failure behavior for session-store unavailability.

1 post · 2026

Timeline

Feature trailActive

AI Model Attention Co-Design for Inference

This feature thread tracks the evolution of co-designing AI model attention mechanisms with hardware execution characteristics to optimize inference performance. Initial efforts focused on understanding the interplay between architectural choices like group size, head dimension, and sequence length, and their impact on compute-bound prefill and memory-bound decode phases. Subsequent developments have introduced practical guidelines and optimizations for NVIDIA hardware, including strategies for hardware-friendly dimensioning of linear layers in LLMs, NVFP4 quantization, and expert parallelism (EP) and hybrid parallel strategies like pipeline parallelism and Helix Parallelism in TensorRT-LLM for scaling large Mixture-of-Experts models across multi-node Blackwell NVLink systems.

4 posts · 2026

Timeline

Feature trailActive

GPU Sizing for AI Inference

This feature thread tracks the evolution of methodologies and tools for accurately sizing GPU infrastructure for AI inference workloads, focusing on optimizing performance and Total Cost of Ownership (TCO). Initial efforts focused on mapping inference workloads to use-case categories and identifying key sizing inputs such as model selection, DAUs, concurrency, token lengths, cache hit rate, and latency targets. Subsequent developments have introduced a core-and-flex capacity model, emphasizing r

1 post · 2026

Timeline

Feature trailActive

World Action Models for Robot Manipulation

This feature thread tracks the evolution of World Action Models (WAMs) for robot manipulation, focusing on their ability to generalize beyond training data by learning world dynamics. Initial efforts focused on the limitations of Vision-Language Models (VLMs) in robotics, which excel at semantic understanding but lack predictive dynamics. WAMs, built on video world models, overcome this by learning how the world evolves, enabling better physical generalization, adaptation to new robots with fewer training examples, and the development of NVIDIA Cosmos 3. Cosmos 3 unifies physical reasoning, world generation, and action generation within a single Mixture-of-Transformers (MoT) architecture, featuring a Reasoner tower (VLM) and a Generator tower (diffusion-based). It supports multiple modalities and offers Nano (16B) and Super (64B) model checkpoints, along with open datasets and deployment tools (Cosmos NIM microservices).

8 posts · 2026

Timeline

Feature trailActive

Omniverse NuRec for AV Perception Adaptation

This feature thread tracks the evolution of NVIDIA Omniverse NuRec for reconstructing real-world driving scenes and rendering new camera views for target vehicle configurations. Initial efforts focused on enabling novel-view synthesis and reusing scene data for adaptation. Subsequent developments have packaged the workflow as agent skills for downloading scenes, rendering with NuRec, and applying Harmonizer for temporal consistency. This post details the process of adapting perception models to specific vehicle configurations and optimizing the reconstruction pipeline using NVIDIA Nsight Developer Tools. Optimizations focused on kernel fusion, reducing synchronization, and improving GPU occupancy, leading to significant runtime reductions in the neural reconstruction pipeline.

1 post · 2026

Timeline

Feature trailActive

Omniverse RTX Sensor Simulation SDK

This feature thread tracks the evolution of integrating NVIDIA Omniverse RTX sensor simulation capabilities into existing applications. Initial efforts focused on providing modular APIs, like ovrtx, within the NVIDIA Agent Toolkit. ovrtx acts as a lightweight C/Python SDK enabling applications to configure render and sensor outputs (camera, lidar, radar) from OpenUSD scenes, step simulation from the app loop, and retrieve output tensors. It integrates with other Omniverse libraries (ovphysx, ovstream, ovstorage, ovstage) for multi-modal simulation and visualization pipelines essential for synthetic data generation, digital twins, and physical AI validation in advanced 3D and industrial domains.

2 posts · 2026

Timeline

Feature trailActive

Fast Startup for Inference Workloads on Kubernetes

This feature thread tracks the evolution of enabling fast startup for AI inference workloads on Kubernetes. Initial efforts focused on the cold-start problem, where initializing inference replicas can take several minutes, leading to idle GPUs and potential SLA violations during traffic spikes. NVIDIA Dynamo Snapshot addresses this by leveraging CRIU and cuda-checkpoint for near-instant checkpoint/restore of single-GPU AI inference workloads. Key optimizations include quiesce/resume hooks for workloads, KV cache unmapping, parallel memfd restore, Linux native AIO, and a GPU Memory Service (GMS) for decoupling model weights. Experimental results show significant startup time reductions for large models.

2 posts · 2026

Timeline

Feature trailActive

High-Performance Core Math Libraries in Python

This feature thread tracks the evolution of providing high-performance core math operations to Python developers, enabling them to leverage NVIDIA's accelerated math libraries (CUDA-X, NVPL) for CPU, GPU, and distributed systems. Initial efforts focused on bridging the gap between Python's scientific ecosystem and low-level GPU acceleration. Subsequent developments have introduced libraries like nvmath-python, offering a Pythonic abstraction layer over libraries such as cuFFT, cuBLASLt, cuDSS, cuSPARSE, and cuTENSOR, with support for Universal Sparse Tensors (UST) and flexible installation options. The library allows for choice of memory and execution spaces, and provides both generic and specialized APIs for various use cases. Logging capabilities are integrated for tracing computational details and data flow. Composite operations are highlighted for their importance in performance optimization.

2 posts · 2026

Timeline

Feature trailActive

Spectrum-X Ethernet for AI Factories

This feature thread tracks the evolution of Spectrum-X Ethernet, a hardware-accelerated networking architecture designed for giga-scale AI factories. Initial efforts focused on addressing the limitations of traditional Ethernet for AI workloads, such as hash collisions, stragglers, lossy operation, and slow congestion control. Spectrum-X Ethernet introduces co-designed high-performance switches and SuperNICs to deliver predictable low latency, high fabric utilization, and robust resilience. It implements three hardware-accelerated control loops: in-switch Adaptive Routing, targeted Congestion Control, and NIC-based Plane Load Balancing. Spectrum-X Multiplane technology decomposes a single host's bandwidth into multiple physically independent two-tier fat-tree planes, enabling scaling to over 128,000 endpoints without the latency and jitter of traditional multi-tier topologies. The hardware-accelerated Plane Load Balancer maintains per-plane congestion state and local queue depth to isolate failures, achieving failover in 2.68 milliseconds versus 1.08 seconds for traditional Ethernet. Under a 10% fabric link failure scenario, Spectrum-X Ethernet degrades bandwidth by a proportional 11% and increases tail latency by only 7%, while traditional Ethernet collapses by 50% or more.

1 post · 2026

Timeline

Feature trailActive

GPU-Accelerated Financial Clustering

This feature thread tracks the evolution of GPU-accelerated clustering techniques for financial instruments at scale. Initial efforts focused on developing AdaptGrow, a GPU-accelerated symmetric non-negative matrix factorization (SymNMF) solver capable of producing hard cluster labels, soft factor loadings, and structural-break signals from rolling correlation and tail pairwise dependence matrices. AdaptGrow features a memory-efficient formulation reducing peak storage and a distributed implementation row-sharding the dependence matrix and communicating O(nk) data via NCCL, scaling to one million instruments across 64 NVIDIA GB200 GPUs on 16 nodes. AdaptGrow auto-configures from the matrix eigenspectrum, choosing full-batch AdaGrad for clean spectral gaps and block-stochastic SVRG updates for flatter spectra, using the same solver interface from one GPU to many. Across 250 rolling windows approximating one trading year, adjusted Rand index (ARI) stability tracking with a self-calibrated 3σ control limit detects planted membership changes, while TPDM cross-sector dependence exposes co-crash events that correlation misses. Spherical k-means provides a lower-cost hard-clustering baseline for well-separated sectors, while SymNMF retains graded factor loadings when boundaries are ambiguous or the spectrum collapses toward a dominant factor.

1 post · 2026

Timeline

Feature trailActive

Transaction Foundation Models for Financial Intelligence

This feature thread tracks the evolution of building and deploying transaction foundation models for financial intelligence. Initial efforts focused on leveraging transformer-based models pre-trained on large volumes of unlabeled transaction sequences to produce general-purpose representations of financial behavior that transfer across a wide array of downstream tasks, including fraud detection, credit scoring, and personalized recommendations. Subsequent developments have introduced a modular workflow for agentic systems, enabling automated signal discovery through specialized agents for signal generation, code translation, and iterative evaluation. This includes the use of NVIDIA Nemotron models and the NeMo Agent Toolkit, with config-driven workflows and built-in observability via Arize Phoenix tracing.

2 posts · 2026

Timeline

Feature trailActive

Token-Metered AI Service Billing and Management

This feature thread tracks the evolution of building and operating token-metered AI services, focusing on the economic models, technical infrastructure, and software components required to transition from GPU-hour billing to token-based consumption. It covers the development of AI factories, the integration of AI developer studios and marketplaces, and the implementation of robust metering and billing systems that track token usage, performance, reliability, and governance KPIs. The goal is to enable telcos to monetize AI infrastructure as token factories, moving up the value chain from compute-as-a-service to token-as-a-service. This post details the architectural components and KPIs necessary for this transition, including the shift from selling GPU hours to delivering token-metered AI services, the role of AI developer studios and marketplaces, and the importance of token-level metering and billing for performance, governance, and economic optimization.

2 posts · 2026

Timeline

Feature trailActive

GPU-Accelerated Semiconductor Materials Simulation and Manufacturing

This feature thread tracks the evolution of GPU-accelerated platforms for semiconductor innovation, encompassing atomic-scale materials discovery, process engineering, and factory optimization. Initial efforts focused on leveraging NVIDIA CUDA-X libraries like cuDSS and cuEST to accelerate materials simulation and density functional theory workflows, achieving significant speedups in quantum chemistry and chamber simulation times. Subsequent developments have integrated these capabilities into a comprehensive end-to-end digital development model, combining Applied Materials' Ginestra and ACE+ platforms with NVIDIA's CUDA-X libraries (cuDSS, cuEST, PhysicsNeMo) and Omniverse for digital twins. This collaboration accelerates the entire innovation pipeline, enabling faster exploration of material-property-performance relationships, optimization of process recipes, and virtual validation of fab-wide operational strategies.

3 posts · 2026

Timeline

Feature trailActive

Multi-GPU UMAP for Massive Datasets

This feature thread tracks the evolution of enabling massive-scale UMAP (Uniform Manifold Approximation and Projection) for dimensionality reduction on datasets with tens to hundreds of millions of vectors, leveraging multi-GPU parallelism. Initial efforts focused on single-GPU UMAP and out-of-core approaches for datasets exceeding GPU memory. Subsequent developments introduced multi-GPU support for the critical all-neighbors kNN graph construction step, allowing for end-to-end scaling and dramatic reductions in training runtimes. This includes the introduction of parameters like `knn_n_clusters` and `knn_overlap_factor` to manage the trade-off between space, time, and quality, and the integration with the cuVS all-neighbors API for standalone graph construction.

1 post · 2026

Timeline

Feature trailActive

GPU Observability for Kubernetes

This feature thread tracks the evolution of GPU observability solutions for Kubernetes environments. Initial efforts focused on addressing the gap in standard Kubernetes metrics for GPU utilization, leading to the development of tools like the GPU Usage Monitor. This project integrates DCGM Exporter, kube-state-metrics, Prometheus, and Grafana into a single, easy-to-deploy Helm chart. The goal is to provide real-time visibility into GPU allocation, compute utilization, memory consumption, and pod status across an entire Kubernetes cluster and through a single Helm chart deployment. The latest iteration focuses on operational simplicity, enabling actionable GPU visibility within minutes via a single Helm install command, complete with pre-built dashboards designed specifically for GPU-accelerated workloads.

2 posts · 2026

Timeline

Feature trailActive

Agentic Video Skills for Jetson

This feature thread tracks the evolution of agentic video skills for NVIDIA Jetson platforms, enabling programmable, device-aware video workflows. Initial efforts focused on providing foundational agentic skills above the Video Codec SDK and PyNvVideoCodec, connecting developer intent to live device discovery, supported configurations, reproducible execution, and performance verification. JetPack 7.2.1 introduces foundational video workflows through the unified jetson-videosdk skill, allowing developers to automate tasks like Linux customization, memory optimization, and model benchmarking. JetPack 7.2 also introduces Multi-Instance GPU (MIG) support on Jetson Thor for isolated, predictable multiworkload execution, critical for real-time AI applications in robotics and industrial automation. Official Yocto Project support and Super Mode for Jetson AGX Orin 32 GB further enhance system efficiency and AI performance.

1 post · 2026

Timeline

Feature trailActive

Closed-Loop Autonomous Vehicle Model Training

This feature thread tracks the evolution of closed-loop training for autonomous vehicle (AV) models, enabling them to learn from the consequences of their own actions in simulation. Initial efforts focused on bridging the gap between open-loop training and real-world deployment by introducing frameworks like NVIDIA Alpamayo, AlpaSim, and AlpaGym. This post details the process of using AlpaGym to post-train AV models in a closed-loop reinforcement learning setup, connecting simulator feedback directly to the policy training loop. It covers installation, reward definition, and launching training, extending the capability to refine policies based on simulated environmental interactions.

1 post · 2026

Timeline

Feature trailActive

NVIDIA Vera Storage Processor Performance

This feature thread tracks the evolution of NVIDIA's storage processing capabilities, focusing on the performance and efficiency gains provided by the NVIDIA Vera BlueField-4 STX Storage Processor. Initial efforts focused on understanding the demands of AI-native storage for agentic AI workloads, which require high throughput for operations like encryption, compression, integrity checking, and data recovery. Subsequent developments have demonstrated the Vera CPU's ability to significantly outperform x86 CPUs in agentic workload execution, particularly in sandboxed code execution, data retrieval, and orchestration tasks. The Vera CPU's design emphasizes high per-core performance, high concurrency, and energy-efficient memory bandwidth, making it a critical component for scaling agentic AI and reinforcement learning in AI factories. The Olympus core, with its advanced branch prediction and out-of-order execution, coupled with a high-bandwidth LPDDR5X memory subsystem and the NVIDIA Scalable Coherency Fabric, delivers superior performance for these demanding workloads.

3 posts · 2026

Timeline

Feature trailActive

NVIDIA Video Codec SDK Enhancements

This feature thread tracks the evolution of the NVIDIA Video Codec SDK, focusing on advancements in video encoding, decoding, and transcoding capabilities. SDK 13.1 introduces AV1 Hierarchical Reference Mode with up to 31 B-frames for enhanced encoding efficiency, per-macroblock decode statistics for H.264/HEVC to enable GPU-accelerated video analytics, and frame-accurate seek functionality for AI and editing workflows. The transcoder pipeline has been redesigned with a modular, queue-based architecture and zero-copy transcode support with shared CUarray buffers, resulting in lower SM and memory bandwidth utilization, higher throughput, greater customizability, and simplified deployment using an official Docker-based development environment.

1 post · 2026

Timeline

Feature trailActive

GPU-Native Medical Physics Simulation for Healthcare Robotics

This feature thread tracks the evolution of GPU-native medical physics simulation frameworks for healthcare robotics. Initial efforts focused on addressing data scarcity, generalization challenges, and slow development velocity in medical robotics by providing realistic, high-fidelity simulation environments. The NVIDIA Medical Physics Simulation framework, integrated within NVIDIA Isaac for Healthcare, leverages GPU acceleration (NVIDIA Warp, Newton Physics, CUDA) for real-time simulation of de. Subsequent developments have introduced NV-Generate-CTMR and NV-Generate-MR-Brain, open-source frameworks for synthesizing high-resolution 3D CT and MRI volumes with pixel-level anatomical segmentation. These frameworks enable privacy-preserving data augmentation, research, and accelerate downstream medical imaging AI development by providing realistic, anatomically consistent 3D volumes, often paired with segmentation labels, to help models generalize better when labeled examples are scarce and enable consistent benchmarking. NV-Generate-CTMR supports flexible voxel sizes, variable volume dimensions, and whole-body coverage within a single model, acting as a foundation model for medical imaging adaptable to many downstream tasks and anatomies. MAISI-v2, based on Latent Rectified Flow, achieves state-of-the-art image quality with much faster inference compared to prior medical image generation methods.

2 posts · 2026

Timeline

Feature trailActive

Quantum Error Correction Decoders for Color Codes

This feature thread tracks the advancement of quantum error correction (QEC) techniques, specifically focusing on improving the efficiency and accuracy of decoders for various QEC codes. Initial efforts explored general QEC concepts and the challenges of implementing fault-tolerant quantum computation. Subsequent developments have concentrated on specific code families like surface codes and, more recently, color codes, addressing the critical need for fast and accurate decoders. This post details the NVIDIA Ising Decoder ColorCode 1 Fast, which demonstrates over 347.7x better logical error rate (LER) and 7.3x faster runtime than Chromobius for d=31 and a physical error rate of 0.3%, reviving color codes as a practical choice for fault-tolerant quantum computation. The Ising Decoding pipeline leverages 3D convolutional neural network (CNN) pre-decoders for triangular color codes, enabling scalable, low-latency, and accurate real-time decoding that can be tuned for specific quantum processor unit (QPU) architectures and noise profiles. NVIDIA provides open access to the Ising model family, including weights, training recipes, synthetic data generation tools using cuQuantum and cuStabilizer, and a full training pipeline, allowing researchers and developers to tailor and deploy high-performance decoders for their own QPUs.

1 post · 2026

Timeline

Feature trailActive

OptiX Toolkit Debugging Utilities

This feature thread tracks the evolution of debugging utilities for NVIDIA's OptiX ray tracing engine. Initial efforts focused on providing robust error-checking macros and device-side debug printing mechanisms to address common debugging challenges in GPU ray tracing. This includes unified, minimal-macro error checking across OptiX, CUDA runtime, and CUDA driver APIs, as well as targeted, interactive device-side debug output with features like debug index matching and visual markers.

1 post · 2026

Timeline

Feature trailActive

Path Tracing in Game Engines

This feature thread tracks the evolution of integrating path tracing into real-time game engines, moving beyond traditional ray tracing for enhanced global illumination and realistic lighting. Initial efforts focused on enabling path tracing for both direct and indirect lighting, improving visual consistency between gameplay and cutscenes, and optimizing for denoising technologies like DLSS Ray Reconstruction. Subsequent developments have involved custom implementations for specific rendering pipelines, and the introduction of DLSS 4.5 with Dynamic Multi Frame Generation and a second-generation transformer model for Super Resolution, now available as an Unreal Engine plugin. The NVIDIA RTX Branch of Unreal Engine (NvRTX) also receives stability-focused updates.

3 posts · 2026

Timeline

Feature trailActive

Observability and Control for TensorRT Engine Builds

This feature thread tracks the evolution of making NVIDIA TensorRT engine builds more observable and controllable. Initially, long-running builds lacked visibility, leading to wasted GPU hours and user frustration. Recent advancements, exemplified by the introduction of `IProgressMonitor` in TensorRT, provide fine-grained, thread-safe progress tracking and cancellation capabilities. This allows developers to integrate real-time progress reporting into terminals, IDEs, services, and agent runtimes, and decouples build logic from application-level rendering or streaming.

1 post · 2026

Timeline

Feature trailActive

Multi-Camera 3D Tracking and Calibration with DeepStream

This feature thread tracks the evolution of multi-camera 3D object tracking and the associated camera calibration challenges within NVIDIA DeepStream. Initial efforts required significant manual effort for camera calibration and complex calculations for consistent object tracking across multiple views. Recent advancements, exemplified by DeepStream 9.1, introduce automated calibration (AutoMagicCalib - AMC) and a robust Multi-View 3D Tracking (MV3DT) skill, further enhanced by agentic skills for natural language-driven pipeline deployment. MV3DT fuses detections from multiple auto-calibrated cameras into a shared 3D coordinate system and maintains a consistent object ID across views, processing detections from models like PeopleNetTransformer, PeopleNet v2.6.3, and RT-DETR 2D. AMC automates camera calibration by analyzing tracked objects moving across existing video files or streams, estimating intrinsic and extrinsic parameters. The system supports on-screen display, bird's-eye view, and Kafka messaging outputs.

1 post · 2026

Timeline

Feature trailActive

AI-Assisted USD Runtime Development

This feature thread tracks the evolution of developing Universal Scene Description (USD) runtimes. Initially, building USD implementations often required adapting large, legacy codebases, which was time-consuming and inflexible for specific memory, performance, or ABI constraints. Recent advancements, exemplified by nanousd-labs, introduce AI agents that can generate lightweight, spec-compliant USD runtimes directly from the USD Core Specification. This approach bypasses the need to adapt existing codebases and allows for tailored memory, performance, and ABI constraints, with compliance validated against specification-derived test suites. Developers can use nanousd directly or adapt the agent-driven methodology to build or validate custom physical AI workflows.

1 post · 2026

Timeline

Feature trailActive

Hardware Acceleration for Binary Field Arithmetic in CUDA

This feature thread tracks the development of hardware-accelerated binary field arithmetic within NVIDIA CUDA. Initial efforts focused on the challenges of emulating operations like carryless multiplication, crucial for cryptography and coding theory, with significant overhead. The introduction of the `clmad` instruction in CUDA 13.3 on Ampere and newer GPUs marks a significant advancement, providing hardware acceleration for these operations. This has led to substantial performance improvements in cryptographic workloads like GHASH and sum-check protocols, enabling faster and more efficient implementation of authenticated encryption, error-correcting codes, and zero-knowledge proofs on existing NVIDIA hardware.

1 post · 2026

Timeline

Feature trailActive

Nonuniform Tensor Parallelism for LLM Training

This feature thread tracks the evolution of Nonuniform Tensor Parallelism (NTP) for large-scale LLM training. NTP enables training jobs to dynamically adapt their tensor parallelism degree in response to transient GPU unavailability, ensuring sustained Goodput and minimizing computational stalls. It incorporates dynamic power boosting to compensate for resource reductions and uses efficient, overlapped tensor resharding techniques to limit overhead to less than 1%. NTP supports scale-up domains up to 72 GPUs on NVIDIA Blackwell and Blackwell Ultra systems with NVLink interconnects.

1 post · 2026

Timeline

Feature trailActive

Kernel Fusion in CUDA

This feature thread tracks the evolution of kernel fusion techniques in NVIDIA CUDA to optimize GPU performance. Initial efforts focused on manual kernel fusion in CUDA C++ to reduce memory traffic and kernel launch overhead by combining multiple operations into a single kernel. Subsequent developments have explored implicit kernel fusion through compilers like PyTorch's Torch Inductor, which automatically fuses operations like element-wise transformations and reductions. This post details how custom fused MLP kernels, built with the CuTe DSL, address MoE bottlenecks by eliminating memory and synchronization overhead, enabling sync-free MoE execution with full-iteration CUDA graphs and achieving 1.3x-2x kernel-level speedup over unfused paths. These fused kernels support advanced GLU activation functions and quantization, and integrate with cuDNN Frontend, Transformer Engine, and Megatron Core, delivering significant end-to-end throughput improvements in pre-training setups.

2 posts · 2026

Timeline

Feature trailActive

GPU-Accelerated Presto for Low-Latency Analytics

This feature thread tracks the evolution of GPU-accelerated query engines, focusing on optimizing data layout, transfer orchestration, and compression for low-latency analytics. Initial efforts demonstrated significant performance gains with GPU-accelerated Presto on NVIDIA DGX B200 systems. Subsequent developments, exemplified by NVIDIA GQE, introduce a reference architecture leveraging modern NVIDIA hardware like HBM, NVLink-C2C, and dedicated decompression engines in GB200 NVL72. This post details GQE's architecture, data layout, and transfer optimizations, including hybrid compression strategies and pipelined data movement, achieving substantial speedups on benchmarks.

2 posts · 2026

Timeline

Feature trailActive

AI-Native RAN for Spectral Efficiency

This feature thread tracks the evolution of AI-Native Radio Access Networks (RAN) to maximize spectral efficiency. Initial efforts focused on overcoming compute constraints in traditional RANs by leveraging GPU acceleration for mathematically dense Layer 1 and Layer 2 models. This enables significant improvements in massive MIMO scenarios through AI-based beamforming and Deep Reinforcement Learning (DRL) for link adaptation. Field trials demonstrate substantial throughput gains and the potential for monetization of underutilized GPU resources. This post details how NVIDIA AI Aerial leverages GPU-accelerated, AI-native architectures to overcome prior compute constraints, enabling mathematically dense Layer 1 and Layer 2 models that significantly increase spectral efficiency, particularly for massive MIMO scenarios. Field trials and engineering results demonstrate that NVIDIA AI Aerial achieves up to 1.62x throughput gains in AI-based beamforming and 1.3x throughput improvements in DRL-based link adaptation compared to traditional CPU-based methods, with practical support for large models and low-latency inference critical for real-world deployment. The platform supports scalable, software-defined RAN evolution enabling dynamic model growth, cross-cell coordination, integrated sensing and communication, and monetization of underutilized GPU resources, facilitating AI-native 5G-Advanced and 6G networks in partnership with industry leaders such as Nokia.

1 post · 2026

Timeline

Feature trailActive

Vulkan Descriptor Heaps for Resource Binding

This feature thread tracks the evolution of resource binding in Vulkan, focusing on the introduction and adoption of descriptor heaps. Initial efforts focused on the limitations of descriptor sets, which involved complex layouts, allocation through pools, and cumbersome management. The `VK_EXT_descriptor_heap` extension, now supported by NVIDIA drivers 610+ and Nsight Graphics 2026.2, introduces a user-managed, single-heap model for resource and sampler descriptors. This significantly simplifies resource management, enables dynamic descriptor indexing, and improves performance, bringing greater parity with Direct3D 12. Tooling support includes debugging, inspection, and capture/replay workflows.

2 posts · 2026

Timeline

Feature trailActive

BEV Pooling Optimization for Physical AI

This feature thread tracks the evolution of optimizing Bird's-Eye-View (BEV) pooling for NVIDIA GPUs, crucial for physical AI applications like autonomous vehicles and robotics. Initial efforts focused on the fundamental BEV pooling operation, which gathers depth-weighted image features and scatters them into BEV grid cells. Subsequent developments, exemplified by BEVPoolV3, introduce algorithmic and implementation optimizations to significantly reduce latency. These include reducing duplicate depth loads, a five-array INT32 scatter map, precomputed indices eliminating runtime integer division, and interval-owned output writes, enabling substantial latency reduction across different GPU memory regimes. The workflow for optimizing BEV pooling operators involves classifying the working set by L2 fit, eliminating redundant scatter traffic, mapping kernel implementation to the target NVIDIA GPU architecture, and validating bottlenecks with NVIDIA Nsight Compute, with optimization strategies selected according to whether the workload is DRAM-bound or L2-resident. Benchmarks on NVIDIA RTX A6000 (Ampere) and NVIDIA RTX PRO 6000 Blackwell Max-Q demonstrate that BEVPoolV3 achieves up to 22x speedup (FP16) on the DRAM-bound path and up to 42x speedup (FP8) on the L2-resident path over V2, with FP8 delivering further latency gains for L2-resident scatter-reduce workloads, while edge-class platforms require architecture-specific evaluation for FP8 adoption.

1 post · 2026

Timeline

Feature trailActive

Real-Time AI for High-Speed Data Acquisition

This feature thread tracks the evolution of enabling real-time AI processing for high-speed data acquisition systems. Initial efforts focused on overcoming traditional data acquisition bottlenecks by introducing adaptable, software-centric architectures. NVIDIA DAQIRI, as part of the NVIDIA Holoscan Platform, directly connects high-bandwidth streaming detectors and sensors to the NVIDIA software ecosystem, enabling zero-copy streaming of data to GPU memory for immediate AI processing. This allows for real-time analysis of data streams previously discarded due to bandwidth limitations, as demonstrated by the A-GHOST project at CERN. Subsequent developments focus on optimizing the DAQIRI pipeline for various hardware configurations and integrating it with advanced AI models for tasks like filtering, inference, compression, and adaptive control.

1 post · 2026

Timeline

Feature trailActive

Intent-Based Security Profiles for InfiniBand

This feature thread tracks the evolution of simplifying and standardizing security configuration for NVIDIA Quantum InfiniBand. Initial efforts focused on the need for easy implementation of advanced security capabilities. This post introduces intent-based security profiles (General, Bare Metal Cloud, Secured Bare Metal Cloud) within Unified Fabric Manager (UFM). These profiles automate the configuration of PKey isolation, MAD key protection, GUID-based access control, and continuous validation, significantly reducing manual configuration errors and deployment time. Continuous Security Verification (CSV) provides automated auditing and remediation guidance, giving users a real-time security health score to ensure ongoing protection and compliance across InfiniBand deployments.

1 post · 2026

Timeline

Feature trailActive

Battery Energy Storage Systems for AI Factories

This feature thread tracks the evolution of Battery Energy Storage Systems (BESS) as critical infrastructure for AI factories. Initial efforts focused on understanding the role of BESS in buffering fast-changing, power-dense AI loads, improving power quality, and enabling flexible grid interconnection. Subsequent developments have emphasized integrated design, aligning battery cells, power conversion, telemetry, and control architecture with site-level modeling of computational loads. Rigorous validation frameworks, such as the NVIDIA BESS Self-Qualification Guidelines, are emerging to ensure that BESS solutions support AI-specific requirements, including load buffering, ride-through, telemetry, and operational flexibility, while also addressing manufacturability, scalability, and reliability for large-scale production infrastructure.

1 post · 2026

Timeline

Feature trailActive

Automated AI Model Documentation Generation

This feature thread tracks the evolution of automated AI model documentation generation, focusing on tools and techniques that streamline the creation of comprehensive and auditable model cards. Initial efforts focused on the manual and time-consuming process of creating model documentation. Subsequent developments, exemplified by the NVIDIA Model Card Generator (MCG) toolkit, introduce containerized pipelines that automate documentation by reading directly from source code and associated files. The MCG toolkit operates in three stages: Ingestion, Extraction, and Rendering, leveraging NVIDIA Inference Microservices (NIM) and large language models for high-precision data retrieval, content generation, and formatting. It produces an overview card and four subcards covering Bias, Explainability, Privacy, and Safety & Security. Customization options include configurable language models, templates, and field-level guides, allowing organizations to adapt the toolkit for different compliance needs and industry standards. Performance testing shows the toolkit generates model cards quickly with high completion rates and accuracy, flagging missing information when documentation is sparse. Oracle is an early adopter, integrating the MCG toolkit into its OCI AI infrastructure.

1 post · 2026

Timeline

Feature trailActive

AI-Driven Compiler Auto-Tuning

This feature thread tracks the evolution of AI-driven compiler auto-tuning frameworks, focusing on optimizing NVIDIA GPU compilers for specific workloads. Initial efforts focused on the limitations of generic compiler heuristics, which often fail to achieve optimal performance for critical kernel hotspots. NVIDIA CompileIQ, introduced in CUDA 13.3, leverages evolutionary and genetic algorithms to explore a rich space of internal compiler parameters, generating specialized configurations (ACFs) for individual workloads, surpassing default heuristics in performance tuning. It targets critical kernel hotspots in workloads like LLM inference, enabling fractional performance gains to yield significant overall throughput improvements. CompileIQ supports multi-objective optimization balancing runtime, compile time, and power consumption, producing Pareto-optimal compiler configurations that are reproducible, portable, and secure for production use in AI and HPC environments.

1 post · 2026

Timeline