Inside Prometheus: Meta’s Journey to a Gigawatt-Scale Supercluster

This post details the first phase completion and scaling plans for Prometheus, Meta's gigawatt-scale AI supercluster designed for training foundational models. It highlights the deployment of NVIDIA GB200 and GB300 NVL72 hardware platforms, detailing their GPU, HBM, NVLink, and power specifications. The post also elaborates on the dual-network architecture (standard data center and dedicated Back-End network) for GPU-to-GPU communication, including building-level fabrics (DSF and NSF) and aggregation networks (BAG). It emphasizes engineering reliability at scale, citing goodput metrics (>85% for pre-training, >90% for RL) and mean time between interruptions (MTBI) of ~47k GPU hours, with ongoing improvements to over 54K GPU hours.