Accelerating Facebook’s infrastructure with application-specific hardware

This post introduces Meta's next-generation hardware platform for AI training, code-named "Zion," and new custom chip designs (ASICs) optimized for AI inference ("Kings Canyon") and video transcoding ("Mount Shasta"). Zion is a large-memory unified training platform utilizing OCP accelerator modules (OAM) and a disaggregated design for independent scaling of memory, compute, and network components. Kings Canyon focuses on scalable inference servers using M.2 modules, single-socket servers, carrier cards, and chassis, with support for INT8 and FP16 precision. The post also details the challenges of large model partitioning for inference and the role of the Glow compiler in abstracting vendor-specific hardware for ML accelerators. For video transcoding, custom ASICs (Mount Shasta) are designed to improve efficiency and enable new functionalities like real-time 4K 60fps streaming by dedicating silicon for decoding, scaling, and encoding stages.