Timestep distillation: 2.5x faster FLUX.2 image generation

This post details the application of timestep distillation to FLUX.2 for faster image generation. It explains the core concepts of Distribution Matching Distillation (DMD) and its evolution to DMD2, including distribution-level matching, the two-timescale update rule (TTUR), and the use of a GAN discriminator. Engineering challenges and solutions for scaling DMD2 to FLUX.2 are discussed, focusing on memory optimization through mixed-precision training (BF16), FSDP, and activation checkpointing, and the use of NVIDIA's FastGen framework.