Meta’s Generative Ads Model (GEM): The Central Brain Accelerating Ads Recommendation AI Innovation

This post introduces Meta's Generative Ads Recommendation Model (GEM), a large foundation model for recommendation systems (RecSys) built on an LLM-inspired paradigm. GEM introduces architectural innovations for efficient scaling, including stackable factorization machines with cross-layer attention for non-sequence features, a pyramid-parallel structure for offline sequence modeling, and InterFormer for cross-feature learning that preserves full sequence information. It also employs multi-domain learning to tailor predictions to specific Meta surfaces while leveraging cross-surface insights. Post-training, GEM uses advanced knowledge transfer techniques, including a Student Adapter for refined supervision and representation learning, achieving 2x the effectiveness of standard knowledge distillation. The training infrastructure was re-engineered to deliver a 23x increase in effective training FLOPs using 16x more GPUs, with improved model FLOPS utilization (MFU).