Building Prometheus: How Backend Aggregation Enables Gigawatt-Scale AI Clusters

This post introduces and details Meta's Backend Aggregation (BAG) technology, a new centralized Ethernet-based super spine network layer designed to interconnect multiple spine layer fabrics across data centers and regions for gigawatt-scale AI clusters like Prometheus. It describes how BAG connects thousands of GPUs across different network fabrics (DSF and NSF), detailing planar and spread connection topologies, inter-BAG connectivity, and the connection of BAG layers to L2 fabrics. It also covers hardware specifics (Jericho3 ASICs), routing (eBGP with UCMP), resilience design (port striping, failure domain analysis), and considerations for long cable distances requiring deep buffer switches.