Real-time video generation inference on Baseten

This post details significant performance optimizations for real-time video generation inference on the Wan 2.2 model. Key contributions include: 1) Timestep distillation reducing generation steps from 40-80 to 4, achieving ~20x speedup using Distribution Matching Distillation (DMD2) and sequence parallelism. 2) Custom kernel engineering for Self-Attention bottlenecks, resulting in a 1.58x performance improvement for Video Sparse Attention (VSA) and kernel fusion (Adaptive LayerNorm modulation, RMS Norm, Gated Residual Add) to reduce launch overhead. 3) NVFP4 quantization for linear layers, providing ~4x higher tensor core throughput compared to BF16. These optimizations collectively yield a 53.6x speedup and a 31x cost reduction. The post also outlines scalable inference infrastructure improvements, including optimized cold starts using the Baseten Delivery Network and advanced queuing, and the implementation of custom content guardrails with a trained classifier to prevent NSFW outputs.