Impactful scheduling for GPU clusters

This post details the implementation of a new GPU cluster scheduler at Ai2 that replaces a priority-based system with GPU time budgets, hierarchical fair-share allocation, and a time-slicing contract. The new system aims to improve the 'impact' metric by prioritizing high-value workloads and ensuring full occupancy. Key technical contributions include the design of a hierarchical fair-share algorithm mirroring research program structures with dynamically set budget weights, and the concept of 'allocated occupancy' vs. 'unallocated occupancy' to manage resource charging and preemption. The post also discusses the 'tragedy of the commons' problem in resource allocation and how the new system addresses it by making resource gaming more expensive than honest budget negotiation.