Meta’s Generative Ads Model (GEM): The Central Brain Accelerating Ads Recommendation AI Innovation

What this post added

This post introduces Meta's Generative Ads Model (GEM), a novel foundation model for ads recommendation systems. GEM features an LLM-inspired architecture with innovations in model scaling (4x efficiency), post-training knowledge transfer (2x effectiveness of standard distillation), and training infrastructure (23x effective training FLOPs, 1.43x MFU). Key technical contributions include: scalable architecture with customized attention mechanisms for sequence and non-sequence features, enhanced Wukong architecture for non-sequence feature interaction modeling, pyramid-parallel structure for offline sequence feature modeling, InterFormer for cross-feature learning, and multi-domain learning for surface-specific optimization. Post-training techniques include Student Adapter for knowledge distillation, representation learning, and parameter sharing. The training stack was re-engineered for massive GPU utilization and efficiency.