Introducing Tunix: A JAX-Native Library for LLM Post-Training- Google Developers Blog

What this post added

Introduced Tunix, a new open-source, JAX-native library for LLM post-training. Tunix provides a comprehensive toolkit for aligning models at scale, built for performance on TPUs and integrating with MaxText. It offers a complete suite of algorithms including Supervised Fine-Tuning (SFT) with `PeftTrainer` (supporting full-weight and parameter-efficient tuning like LoRA/QLoRA via qwix), Direct Preference Optimization (DPO) with `DPOTrainer`, and Reinforcement Learning (RL) algorithms like Proximal Policy Optimization (PPO) with `PPOLearner`, Group Relative Policy Optimization (GRPO) with `GRPOLearner`, and Group Sequence Policy Optimization (GSPO-token) with `GSPO-token`. It also includes a `DistillationTrainer` for model compression using logit-based and attention transfer methods. Tunix is available as a PyPI package (`pip install google-tunix`) and includes examples and documentation. The library enables training of Agents that reason using LLMs and interact with external environments. Quantitative results show a ~12% relative improvement in pass@1 answer accuracy on the GSM8K math reasoning benchmark when fine-tuning Gemma 2 2B-IT with Tunix's GRPO implementation.