TechPrufer
Get started
NVIDIA
NVIDIA logoNVIDIA · Feature trail

CUDA Toolkit Enhancements

2 posts · May 2026 – Sep 2026

What this trail covers

This feature thread tracks the evolution of the CUDA Toolkit, encompassing new platform support, architectural previews, enhanced GPU management, and expanded CUDA Python and CCCL functionality. CUDA Toolkit 13.3 introduces support for Windows on Arm, a preview of the NVIDIA Rubin architecture, modernized Multi-Process Service (MPS) V3 for precise GPU partitioning, and CUDA Compute Fabric Transport (CFT) for advanced inter-GPU communication. It also enhances CUDA Python with texture/surface programming and introduces the `clmad` instruction for hardware-accelerated binary field arithmetic. The thread also covers agentic AI skills for translating CUDA tile operations between Python and Rust, demonstrating a new paradigm for developer tooling and code migration.

All NVIDIA trails
  1. Translating CUDA Tile Operations from Python to Rust Using Agentic AI | NVIDIA Technical BlogThis post details the development of an AI agent skill within the TileGym repository to translate cuTile Python and Triton-TileIR kernels into cuTile Rust. It highlights the challenges of explicit specialization in Rust compared to implicit JIT compilation in Python, using a Softmax kernel as an example. The agent's pipeline includes structural verification via IR diffing and functional testing, achieving 99.5% of cuTile Python performance on average for 24 ported operators. The post also introduces the `clmad` instruction in CUDA 13.3 for hardware-accelerated binary field arithmetic, a significant enhancement for cryptography and coding theory workloads.
  2. CUDA Toolkit 13.4 Adds Windows on Arm Support and Greater Control over Shared GPUs | NVIDIA Technical BlogCUDA Toolkit 13.4 adds support for Windows on Arm, extending CUDA application development beyond Linux on Arm platforms. It provides preview functional support for the NVIDIA Rubin architecture (compute capability 107). Multi-Process Service V3 introduces a modernized control layer with a scriptable CLI, named server instances, TOML configuration, and cgroup-integrated GPU memory limits for precise GPU partitioning. CUDA Compute Fabric Transport (CFT) offers a transport-centric API for communication-library developers to move data across NVLink fabric. CUDA Python expands with cuda.core 1.1.0 adding texture and surface programming, NUMA-aware managed memory, and .pyi type stubs, while cuda.compute 1.1 enables ahead-of-time compilation of CCCL algorithms. CCCL 3.4 delivers faster warp-specialized cub::DeviceScan on Blackwell GPUs, single-call APIs across CUB device-wide algorithms, batched warp reductions, and C++ Standard Library parallel algorithms in cuda::std. The release also decouples the CUDA driver from the toolkit and defaults to Coherent Driver-based Memory Management on coherent platforms.
  3. Introducing CUDA Rust: Two Tracks for Writing GPU Kernels | NVIDIA Technical BlogThis post introduces CUDA Rust, enabling GPU kernel development directly in Rust. It presents two distinct programming models: SIMT (Single Instruction, Multiple Threads) via the `cuda-oxide` codegen backend, which compiles Rust kernels to PTX using Pliron IR and LLVM, and Tile-based programming via `cutile-rs`, which uses CUDA Tile IR for JIT compilation. Both approaches enforce memory safety at compile time: `cuda-oxide` uses `DisjointSlice` and launch contracts, while `cutile-rs` uses tensor partitioning and ownership. The post details the setup, usage, and safety guarantees of each track, noting `cutile-rs`'s lighter requirements (stable Rust) and existing adoption in projects like HuggingFace's Grout.
  4. The Modern CUDA Toolbox in Practice: A Step-by-Step Optimization Walkthrough | NVIDIA Technical BlogThis post demonstrates a step-by-step optimization of a CUDA image processing pipeline using modern CUDA C++ features. It introduces the CCCL API for safer kernel launches and indexing, CUB primitives for efficient block and device-level algorithms, pooled containers (cuda::device_buffer, cuda::host_buffer) to reduce memory allocation overhead and accelerate transfers, and per-thread streams for overlapping kernel execution and data transfers. The optimizations result in a 300x speedup from the initial 6.8s to ~25ms, significantly improving performance and code safety.
  5. CUDA Python 1.0: Stable APIs, One Foundation, Full Platform Access | NVIDIA Technical BlogThis post announces CUDA Python 1.0, marking a significant milestone in making Python a first-class, supported language for the CUDA platform. It introduces stable APIs with semantic versioning guarantees for core components like `cuda.core` (Pythonic CUDA runtime access), `cuda.compute` (CCCL parallel algorithms callable from Python), and `nvmath-python` (NVIDIA math libraries). This unification provides a common foundation for GPU libraries in Python, improving inter-library compatibility and resource sharing. It also highlights the integration of Numba and the CUDA MLIR backend for Python kernel authoring and `cuda.bindings` for comprehensive CUDA C API coverage, aiming to simplify development and reduce version conflicts across the Python GPU ecosystem.
  7. Kernel Fusion in NVIDIA CUDA: Optimizing Memory Traffic and Launch Overhead | NVIDIA Technical BlogThis post details kernel fusion techniques in NVIDIA CUDA to optimize GPU performance. It explains how combining multiple GPU operations into a single device kernel reduces memory traffic and kernel launch overhead. The post provides examples of manual kernel fusion by rewriting two kernels into one `sum_abs_kernel`, demonstrating a 3x speedup and reduced global memory movement. It also discusses implicit kernel fusion using PyTorch's `torch.compile` and Torch Inductor, showing how the compiler can generate fused kernels by performing operations like `abs` inline within a reduction kernel, eliminating intermediate buffers and improving performance.
  8. Develop High-Performance GPU Kernels in C++ with NVIDIA CUDA Tile | NVIDIA Technical BlogThis post introduces CUDA Tile C++, enabling developers to write tile-based GPU kernels directly within existing C++ codebases. It abstracts low-level GPU parallelism, memory movement, and hardware features across NVIDIA architectures. The model uses multi-dimensional tensor spans and partition views to operate on fixed-size array tiles, allowing for more declarative expression of parallel computations like vector addition and matrix multiplication. Optimizations include __restrict__ pointer qualifiers, 16-byte alignment assumptions, and masked load/store operations. Kernels are launched with a single thread per block, with the compiler handling thread execution. Profiling is supported via Nsight Compute. Matrix multiplication kernels leverage tile partitions and MMA operations. Requires compute capability 8.x+, Driver R580+, and CUDA Toolkit 13.3+.
  9. NVIDIA CUDA 13.3 Enhances GPU Development with Tile Programming in C++, Compiler Autotuning, and Python Updates | NVIDIA Technical BlogCUDA 13.3 introduces CUDA Tile programming in C++ for high-level, tile-based kernel development, extending support to Compute Capability 9.0 GPUs. CUDA Python 1.0 is released, stabilizing the ecosystem and adding green contexts, process checkpointing, and inter-process sharing. The CCCL Python release 1.0.0 includes `cuda.compute` with host-callable parallel algorithms and `cuda.coop` for warp-wide primitives in Numba CUDA kernels. A new Numba CUDA MLIR backend is introduced as a drop-in replacement for `numba.cuda`, offering faster JIT compile times and improved diagnostics. The release also adds C++23 support in NVCC, expanded DLPack/mdspan interoperability in CCCL 3.3, and updates to math libraries and profiling tools.