Control How Your GPU Shares Work with Green Contexts | NVIDIA Technical Blog

This post introduces CUDA Green Contexts, a new feature available in CUDA 13.1 (Runtime API) and CUDA 12.4 (Driver API). It details how applications can explicitly partition GPU execution resources, such as SMs and workqueues, within a single process. The post provides a concrete example demonstrating how assigning dedicated SMs to a latency-sensitive kernel, while a bulk workload occupies the remaining SMs, reduces critical kernel latency from 0.140 ms to 0.007 ms on a Blackwell GPU. It also explains the programming model changes, showing how to create streams from a green context handle for more explicit work targeting, and highlights that green contexts are opt-in and additive, allowing incremental adoption.