Accelerating Spatio-Temporal Attention for Video Diffusion on TPUs- Google Developers Blog

This post details the acceleration of spatio-temporal attention for video diffusion models on TPUs by implementing sparse attention kernels using JAX and Pallas Splash Attention. It describes the transition from theoretical sparsity to practical end-to-end speedups by optimizing tile traversal strategies, avoiding intra-tile masking, and aligning sparse masks with execution tiles to reduce latency and improve efficiency on TPU v6e chips.