Scaling data ingestion for machine learning training at Meta

This post details the evolution of Meta's data ingestion infrastructure for machine learning training. It introduces a disaggregated Data PreProcessing tier (DPP) that handles data fetching, decryption, feature extraction, tensor conversion, and last-mile transformations. Key optimizations include feature flattening for efficient selective feature reading (2-2.3x efficiency gains), client-side rebatching to support larger batch sizes and improve reader node throughput (20-40% improvement), and the FlatMaps project to optimize memory bandwidth utilization in data reading workloads. These improvements address the mismatch between hardware performance growth and data ingestion demands, and the power constraints in data centers.