Linear programming for fun and profit

This post details the implementation of Modal's resource solver, which uses linear programming (LP) and Google's GLOP solver to manage cloud compute resources. It explains how the solver handles complex demand scenarios, including multiple instance types, GPU fallbacks, and varying user demand, by formulating the problem as an LP. The post highlights the dual mandate of rapid scaling and cost-optimization, and describes how heuristics are used to prune instance types and reduce solve times. It also details the feedback loop where observed scaling limits from cloud providers are incorporated into subsequent solver runs, and how background workers request instances based on the solver's output.