Sequence learning: A paradigm shift for personalized ads recommendations

This post details a paradigm shift in Meta's ads recommendation engine, moving from Deep Learning Recommendation Models (DLRMs) with human-engineered features to sequence learning architectures. The core innovation is the introduction of Event-Based Features (EBFs) which capture sequential user engagement and conversion events directly, replacing aggregated sparse features. The system utilizes new sequence learning architectures, including custom transformer models with attention mechanisms (reducing complexity from O(N*N) to O(M*N)), to process these EBFs. Scaling challenges for variable-length event sequences are addressed through hardware codesign innovations like native PyTorch support for jagged tensors, kernel-level GPU optimizations, and a Jagged Flash Attention module. Further scaling involves increasing sequence lengths using multi-precision quantization and value-based sampling, and enriching event semantics with multimodal content embeddings via customized vector quantization. The system has shown improvements in ad prediction accuracy, leading to higher value for advertisers and increased conversions.