Mixtral of experts | Mistral AI

This post introduces Mixtral 8x7B, a sparse mixture of experts (SMoE) model with open weights. It details the SMoE architecture where a router network selects two experts per layer for each token, enabling a large number of parameters (46.7B total) with a reduced active parameter count (12.9B per token) for efficient inference. Performance benchmarks against Llama 2 and GPT3.5 are provided, along with analysis of bias and multilingual capabilities. An instruction-following variant, Mixtral 8x7B Instruct, is also released. The post also mentions integration with vLLM for efficient inference and deployment via Skypilot.