MTIA 300: Meta’s First Training Chip with Built-in NICs and Communication-Offloading Engines

This post introduces MTIA 300, Meta's first custom AI training and inference accelerator designed for recommendation and ranking models. Key technical contributions include: 1. Integrated networking: Two network chiplets with six custom 800 Gbps RDMA NICs each provide 1.2 TB/s of total I/O bandwidth, eliminating the PCIe bus bottleneck. 2. Communication offloading: 16 dedicated message engines (MEs) with RISC-V cores, NIC interfaces, and near-memory compute (NMC) blocks handle communication independently, delivering over 2.8 TB/s of reduction throughput and achieving near-perfect isolation from the compute grid. 3. Compiled-communication model: The HCCL library compiles collectives into autonomous execution subgraphs dispatched to MEs, removing host involvement after initial instruction copying. 4. Performance improvements: MTIA 300 achieves up to 940 GB/s of communication bandwidth within a rack and demonstrates a 3.9x faster total communication time compared to equivalent GPU clusters for a large production model. The design principles of integrated networking and offloaded collective execution are highlighted as foundational for future AI workloads.