Get Real-Time Visibility into GPU Usage Across Kubernetes Clusters | NVIDIA Technical Blog

This post details the GPU Usage Monitor, an open-source project that deploys a fully integrated GPU observability stack for Kubernetes. It leverages DCGM Exporter, kube-state-metrics, Prometheus, and Grafana, packaged into a single Helm chart for simplified deployment. The monitor provides real-time visibility into GPU allocation, compute utilization, memory consumption, and pod status, addressing common failure modes like over-provisioning and pod starvation. Key features include tracking namespaces and workloads, displaying per-GPU utilization against thresholds, breaking down memory consumption by pod, and showing running vs. pending pod counts. Configuration options for external Prometheus integration, custom resource allocation, and credential management are also highlighted.