How Meta animates AI-generated images at scale

What this post added

This post details the technical optimizations implemented for Meta AI's image animation feature to achieve large-scale, low-latency generation. Key contributions include: halving floating-point precision (float32 to float16/bfloat16) for reduced memory footprint and faster operations; improving temporal-attention expansion by replicating context tensors after cross-attention layers to reduce compute and memory; leveraging DPM-Solver to decrease sampling steps to 15; combining guidance and step distillation to reduce three forward passes per step to one, requiring only eight solver steps with one U-Net pass per step; and implementing PyTorch optimizations like TorchScripting and freezing for graph simplification and constant transformation, later migrating to PyTorch 2.0 for granular component optimization and advanced parallelism. For deployment, a traffic management system was developed to route requests regionally, minimizing network overhead and latency, and a retry-based probing system was implemented to approximate queueing and prevent failures when near capacity.