Building a ubiquitous shared infrastructure using Twine

This post details the evolution and impact of Twine, Meta's cluster management system, in achieving a ubiquitous shared infrastructure. Key contributions include scaling a single control plane to manage one million machines per region, supporting workload-specific customization via host profiles, and introducing the TaskControl API for collaborative workload life cycle management. The adoption of smaller, power-efficient machines has led to significant power and cost savings. The post highlights the shift from siloed machine pools to a unified infrastructure, with twshared hosting 56% of the fleet and projected to cover all compute services by early 2022.