Baseten

Agentic kernels in production

Aug 29, 2026
Agentic kernels in production
SummaryAI, from the source post

Introduced an agentic kernel development framework that automates the identification, generation, and validation of optimized kernels for production AI models. This framework addresses the gap between kernel benchmarks and production deployment by considering workload-specific configurations, downstream dependencies, higher-level computation restructuring, and seamless integration into serving engines. Achieved 42.3% end-to-end latency reduction on Qwen-Image and 15.2% on FLUX.2, along with a 5.5% increase in tok/s on MiniMax M3 through optimizations like pre-packed FP8 scales, fused QKV projection and epilogue, normalization + quantization kernel fusion, bias absorption, and CFG modulation cache.

More from the Baseten AI Model Deployment and Serving Platform trail