Inference Performance Optimization
Run FLUX.1-dev three times faster

Run FLUX.1-dev three times faster

6/18/2025

What this post added

This post details optimizations applied to the FLUX.1-dev diffusion model for faster inference on Modal. It introduces standard optimizations including using the Torch compiler (`torch.compile` with `max-autotune`), fusing QKV projections, and adopting the channels-last memory layout, which collectively achieved a 1.5x speedup. Additionally, it describes the implementation of an approximate caching technique called First Block Caching, adapted from the ParaAttention repo and TEACache paper, which further reduced latency by 2x, leading to a total 3x performance improvement. The post also touches on reducing compile-time costs for `max-autotune`.

Read the original post ↗