Mixture of Experts (MoEs) in Transformers

This post introduces asynchronous routing and load balancing strategies for Mixture of Experts (MoE) models. It details how to manage the dynamic selection and distribution of expert computations across available hardware, aiming to optimize inference throughput and latency. The DeepSeek-R1 model is used as a concrete example to illustrate these techniques.