A Slurm on Kubernetes Implementation for HPC and Large Scale AI | CoreWeave

This post introduces SUNK (Slurm on Kubernetes), an implementation that integrates the Slurm workload manager with Kubernetes. It details how SUNK allows Slurm jobs to run within Kubernetes containers, enabling a unified platform for both HPC and large-scale AI workloads. The post explains the motivation for this integration (client efficiency and maximizing GPU utilization), the features of SUNK (containerized deployment via Helm, Kubernetes integration benefits, and state management), and the core components: Nodeset (a CRD for managing compute nodes), Syncer (for state synchronization between Slurm and Kubernetes), and a custom Kubernetes scheduler. It highlights the architectural diagram and the benefits of running Slurm components as containerized services within Kubernetes.