Making FlashAttention-4 faster for inference

What this post added

This post details contributions to the FlashAttention-4 kernel to improve LLM inference performance, specifically for decode-heavy workloads. Key technical contributions include: 1. Adding support for FP8 inputs, reducing memory and arithmetic bandwidth demands and enabling smaller KV caches. 2. Implementing support for arbitrary KV page sizes and optimizing performance for smaller pages by introducing a `cpasync`-based path to handle irregular memory accesses, repurposing an idle warp for this task. These changes leverage techniques like split KV parallelism and optimized memory access patterns to boost throughput.