Speeding up GPU kernels by 38% with a multi-agent system

Details a multi-agent system that autonomously optimized CUDA kernels, achieving a 38% geomean speedup by building and optimizing Blackwell GPU kernels from scratch. The system generated 235 optimization problems from production open-source models and achieved significant performance improvements, with 19% of optimizations exceeding 2x. Specific examples include BF16 Grouped Query Attention with Paged Prefill, NVFP4 MoE Linear with Gating, and BF16 Matrix Multiplication, showcasing the system's adaptability and ability to discover novel optimization strategies.