GPU Resource Partitioning with Green Contexts

This feature thread tracks the evolution of GPU resource management and partitioning within a single process. Initial efforts focused on addressing the challenges of concurrent workloads interfering with each other on shared GPU resources, leading to unpredictable performance and increased latency. The introduction of CUDA Green Contexts in CUDA 12.4 (Driver API) and CUDA 13.1 (Runtime API) provides explicit control over GPU execution resources like SMs and workqueues. This allows applications to partition dedicated subsets of SMs and provision workqueues independently, enabling latency-sensitive kernels to run without being blocked by bulk workloads. This significantly reduces critical kernel latency and improves overall GPU utilization by isolating competing tasks.