Scaling data ingestion for machine learning training at Meta

What this post added

This post details the evolution of Meta's data ingestion infrastructure for machine learning training. It highlights the growing demands driven by increased model complexity, data volume, and training throughput. Key contributions include the development of a disaggregated Data PreProcessing tier (DPP) to scale data ingestion and training hardware independently, the introduction of 'feature flattening' for more efficient selective feature reading (yielding 2-2.3x data reading efficiency), and optimizations for memory consumption and bandwidth in the data reading tier, including client-side rebatching and the 'FlatMaps' project. The post also emphasizes the impact of data center power constraints on training accelerator deployment and the need to optimize ingestion to avoid becoming an ingestion-bound bottleneck.