TechPrufer
Get started
NVIDIA
NVIDIA logoNVIDIA · Feature trail

Modern C++ Runtime for CUDA

5 posts · May 2026 – Sep 2026

What this trail covers

This feature thread tracks the evolution of modern C++ abstractions for fundamental CUDA programming model concepts, aiming to make CUDA C++ development safer and more convenient. Initial efforts focused on providing idiomatic C++ APIs for stream management, memory allocation, and kernel launches, leveraging modern C++ features and explicit dependency management. Subsequent developments have introduced the CCCL runtime, offering a collection of headers like `<cuda/stream>`, `<cuda/buffer>`, and `<cuda/launch>`, providing owning types and non-owning refs for resource management and improved composability with existing code.

All NVIDIA trails
  1. Introducing CUDA Rust: Two Tracks for Writing GPU Kernels | NVIDIA Technical BlogIntroduces two new tracks for writing CUDA GPU kernels in Rust: `cuda-oxide` (a custom rustc codegen backend compiling to PTX via Pliron IR and LLVM) and `cutile-rs` (enabling tile-based GPU programming with JIT compilation via CUDA Tile IR). Both projects enforce memory safety at compile time. `cuda-oxide` requires a nightly toolchain and LLVM, while `cutile-rs` works with stable Rust and CUDA 13.3. Plans include inter-language interoperability between CUDA Rust, C++, and Python.
  2. The Modern CUDA Toolbox in Practice: A Step-by-Step Optimization Walkthrough | NVIDIA Technical BlogThis post demonstrates practical optimizations for a CUDA image processing pipeline by incrementally adopting modern CUDA C++ features. It shows how to replace manual CUDA calls with CCCL APIs for safer indexing and stream management, leverage CUB for efficient block-level primitives, utilize pooled memory containers (cuda::device_memory_pool_ref) to reduce allocation overhead, employ pinned host buffers (cuda::host_buffer) for faster host-to-device transfers, and achieve significant speedups by parallelizing GPU work with per-thread streams and asynchronous transfers, reducing the overall pipeline runtime from seconds to milliseconds.
  3. CCCL Runtime: A Modern C++ Runtime for CUDA | NVIDIA Technical BlogThis post introduces the CCCL runtime, a new set of idiomatic C++ APIs available starting in CUDA 13.2 that implement core CUDA functionality: stream management, memory allocation, kernel launches, and more. It contrasts with the traditional CUDA runtime API by using dedicated types (e.g., `cuda::device_ref`, `cuda::stream`) instead of raw identifiers, making dependencies explicit and enabling local reasoning. It also introduces owning types (e.g., `cuda::stream`) and non-owning reference types (e.g., `cuda::stream_ref`) for resource management, following the `std::string` and `std::string_view` pattern, and discourages the use of the default stream by not exposing it in the API.
  4. Develop High-Performance GPU Kernels in C++ with NVIDIA CUDA Tile | NVIDIA Technical BlogThis post introduces CUDA Tile C++, an extension of the CUDA Tile programming model to C++. It allows developers to write tile kernels in C++ using multi-dimensional tensor spans and partition views, abstracting low-level GPU parallelism, memory movement, and hardware features. The post provides examples of vector addition and discusses optimizations like __restrict__ pointer qualifiers and 16-byte alignment, as well as profiling with Nsight Compute. It requires GPUs with compute capability 8.x or newer, NVIDIA Driver R580+, and CUDA Toolkit 13.3+.
  5. NVIDIA CUDA 13.3 Enhances GPU Development with Tile Programming in C++, Compiler Autotuning, and Python Updates | NVIDIA Technical BlogCUDA 13.3 introduces CUDA Tile programming in C++ for high-level, tile-based kernel development, automating parallelism and memory management for portability. CUDA Python 1.0 is released, stabilizing the ecosystem with features like green contexts (partitioning SMs for latency-sensitive kernels), process checkpointing (snapshotting and restoring CUDA state), and inter-process sharing (IPC) for GPU memory. CCCL Python 1.0 adds host-callable parallel algorithms and supports Python lambdas as operators. A new Numba CUDA MLIR backend is introduced, offering a faster JIT compile experience and improved diagnostics for Python GPU programming.