Mixture-of-Kittens: our open-source MoE megakernel for NVL72s

Introduced Mixture-of-Kittens (MoK), a production MoE training megakernel for NVL72s. MoK fuses MoE communication and computation into a single, fully deterministic kernel, addressing the bottleneck of MoE layers in large model training. It achieves significant performance improvements by optimizing for NVL72 hardware, including overlapping computation and communication, choosing the right communication direction (push vs. pull), and minimizing CPU-GPU synchronization. MoK powers Composer training across tens of thousands of GPUs.