
6/18/2025
What this post added
This post details optimizations applied to the FLUX.1-dev diffusion model for faster inference on Modal. It introduces standard optimizations including using the Torch compiler (`torch.compile` with `max-autotune`), fusing QKV projections, and adopting the channels-last memory layout, which collectively achieved a 1.5x speedup. Additionally, it describes the implementation of an approximate caching technique called First Block Caching, adapted from the ParaAttention repo and TEACache paper, which further reduced latency by 2x, leading to a total 3x performance improvement. The post also touches on reducing compile-time costs for `max-autotune`.