In-Language Reasoning: Data Mixing Bridges | Cohere

This post details the development of Tiny Aya L2-Thinker (3.35B), a massively multilingual model capable of in-language reasoning across 60 languages. The key technical contribution is a data mixing strategy comprising three pillars: English reasoning data (ER) for task-solving backbone, multilingual reasoning data (MR) for generalization, and multilingual non-reasoning data (NR) to boost accuracy and enable reasoning in low-resource languages. The model achieves over 93% in-language reasoning rate with minimal accuracy loss on most benchmarks, demonstrating improved cultural reasoning and accessibility.