4/17/2024
What this post added
This post announces and details Mixtral 8x22B, a new open-source SMoE model. It highlights its parameter efficiency (39B active out of 141B), multilingual capabilities, function calling support, and large context window (64K tokens). Performance benchmarks are provided for reasoning, knowledge, multilingual tasks, maths, and coding, comparing it against other open models. The model is released under Apache 2.0.