Serving MiniMax-M3 for efficient inference: Unlocking 1M-Token Context and Multimodality Without Regrets

This post details the engineering breakthroughs and optimizations implemented by Together AI to efficiently serve the MiniMax M3 model. Key contributions include the development of a KV-Block-Major sparse attention kernel, novel paged attention integration for MSA, an optimized index scoring kernel, and a Rust-based multimodal preprocessing gateway. These optimizations resulted in 81-125% throughput improvements and validated Together AI's platform for frontier models with challenges like 1M-token context and native multimodality.