HeyGen x Google Cloud: Bringing Avatar IV to TPUs- Google Developers Blog

This post details the technical efforts to optimize HeyGen's Avatar IV AI video generation model for Google Cloud's TPUs (Trillium v6e). Key contributions include: porting the PyTorch-based model to JAX via torchax, enabling TPU execution; implementing custom attention kernels and sequence parallelism for efficient weight sharding and sequence splitting across an eight-chip mesh; optimizing the super-resolution stage by redesigning sparse attention kernels to eliminate partial blocks and padding by aligning block sizes to frame boundaries; and unchaining the softmax computation by replacing the running maximum with a precomputed bound derived from the Cauchy–Schwarz inequality, significantly reducing serial dependencies in the inner loop. These optimizations resulted in a 1.86x speedup per video chunk, achieving performance comparable to 8x H100 setups with improved cost efficiency.