Mixtral Sparse Mixture of Experts Model
Cheaper, Better, Faster, Stronger | Mistral AI

Cheaper, Better, Faster, Stronger | Mistral AI

4/17/2024

What this post added

This post announces and details Mixtral 8x22B, a new open-source SMoE model. It highlights its parameter efficiency (39B active out of 141B), multilingual capabilities, function calling support, and large context window (64K tokens). Performance benchmarks are provided for reasoning, knowledge, multilingual tasks, maths, and coding, comparing it against other open models. The model is released under Apache 2.0.

Read the original post ↗