The Modern CUDA Toolbox in Practice: A Step-by-Step Optimization Walkthrough | NVIDIA Technical Blog

This post demonstrates practical optimizations for a CUDA image processing pipeline by incrementally adopting modern CUDA C++ features. It shows how to replace manual CUDA calls with CCCL APIs for safer indexing and stream management, leverage CUB for efficient block-level primitives, utilize pooled memory containers (cuda::device_memory_pool_ref) to reduce allocation overhead, employ pinned host buffers (cuda::host_buffer) for faster host-to-device transfers, and achieve significant speedups by parallelizing GPU work with per-thread streams and asynchronous transfers, reducing the overall pipeline runtime from seconds to milliseconds.