JAX Model Evaluation Metrics

2/3/2026

This post introduces Tunix, a JAX-based library for LLM post-training, and demonstrates its use for fine-tuning FunctionGemma on Google TPUs. It details the process of setting up the JAX mesh, applying LoRA adapters with Qwix, creating a custom dataset for completion-only loss, and initiating the training process. The post highlights Tunix's efficient TPU utilization and the resulting accuracy improvements, concluding with instructions for merging LoRA adapters and exporting the fine-tuned model.

11/19/2025

This post introduces the JAX AI Stack, a comprehensive platform for building production AI on Google Cloud TPUs. It details the core libraries (JAX, Flax, Optax, Orbax) and extended ecosystem components (XLA, Pathways, Pallas, Tokamax, Qwix, Grain, MaxText, MaxDiffusion, Tunix, vLLM serving). It highlights the architectural philosophy of modularity and performance, and provides real-world examples of its application in LLM training, generative video, and scientific research. It also points to a technical report and a new central hub (jaxstack.ai) for getting started.

11/13/2025

Introduced Metrax, a new library for performing model evaluation metrics in JAX. Metrax addresses the gap in JAX's native metric support by providing a comprehensive set of predefined metrics for various ML tasks. Key features include parallel computation of 'at K' metrics, iterative merging of metric states for batch processing, and leveraging JAX's `vmap` and `jit` for performance. The library is already in use in Google Search, YouTube, and Tunix, and is open for community contributions.

10/7/2025

Introduces Grain, a flexible data loading library for JAX, and ArrayRecord, a highly efficient file format, to build high-performance data pipelines for large-scale machine learning. Grain ensures efficient data loading and preprocessing with multiprocessing and stateful, checkpointable iterators for reproducibility. ArrayRecord, built on Riegeli, offers efficient random access via a metadata index and parallel I/O through data chunks, enabling true global shuffling and significantly higher read throughput compared to sequential formats like TFRecord. The post also details methods for converting TFRecord datasets to ArrayRecord using `tfds build` or Apache Beam for scalable conversion.

9/30/2025

Introduced Tunix, a new open-source, JAX-native library for LLM post-training. Tunix provides a comprehensive toolkit for aligning models at scale, built for performance on TPUs and integrating with MaxText. It offers a complete suite of algorithms including Supervised Fine-Tuning (SFT) with `PeftTrainer` (supporting full-weight and parameter-efficient tuning like LoRA/QLoRA via qwix), Direct Preference Optimization (DPO) with `DPOTrainer`, and Reinforcement Learning (RL) algorithms like Proximal Policy Optimization (PPO) with `PPOLearner`, Group Relative Policy Optimization (GRPO) with `GRPOLearner`, and Group Sequence Policy Optimization (GSPO-token) with `GSPO-token`. It also includes a `DistillationTrainer` for model compression using logit-based and attention transfer methods. Tunix is available as a PyPI package (`pip install google-tunix`) and includes examples and documentation. The library enables training of Agents that reason using LLMs and interact with external environments. Quantitative results show a ~12% relative improvement in pass@1 answer accuracy on the GSM8K math reasoning benchmark when fine-tuning Gemma 2 2B-IT with Tunix's GRPO implementation.

9/9/2025

This post details how researchers are using JAX's Taylor mode automatic differentiation to solve high-order Partial Differential Equations (PDEs), achieving over 1000x speed-up and 30x memory reduction compared to baseline methods. It highlights the development of the Stochastic Taylor Derivative Estimator (STDE) method, which leverages JAX's ability to efficiently compute arbitrary differential operators in a single forward pass. The post also mentions the generalization of JAX to handle infinite-dimensional vectors (functions in Hilbert space) by describing them as a custom array and registering them with JAX, enabling the calculation of variational derivatives for functionals and operators.

8/12/2025

This post introduces a practical guide to building and pre-training a GPT-2 model using the JAX ecosystem (JAX core, Flax, Orbax, Optax) on TPUs. It provides detailed code examples for defining a Transformer block using NNX, configuring SPMD for distributed training across TPU cores, and implementing a training loop with AdamW optimizer and cosine decay schedule. It also demonstrates how to partition model parameters for potential model tensor parallelism and how to use `jax.device_put` for data parallelism.

7/29/2025

This post details how a robotics researcher leveraged JAX for computational efficiency in optimal control and simulation. Key contributions include the use of JAX's vmap for parallelization and scan for trajectory simulation, achieving significant speedups. It also highlights the seamless integration of model-based and learning-based approaches using JAX's functional paradigm and automatic differentiation, exemplified by the development of the LQRax library for vectorized and differentiable LQR. The post also touches on JAX's growing support for embedded platforms like Jetson, crucial for robotics.

7/16/2025

This post details how the Marin project leverages JAX and its associated frameworks (Levanter, Haliax) to build and train large-scale, reproducible foundation models. Key technical contributions include: 1. Encapsulating training steps into single `@jax.jit`-decorated functions for fused operations and performance optimization. 2. Utilizing `jax.value_and_grad` for efficient loss and gradient computation, and `gradient checkpointing` for memory savings. 3. Employing Pallas-based Splash Attention for optimized Dot Product Attention. 4. Leveraging `@jax.jit` for SPMD parallelization and automated sharding/communication. 5. Introducing Haliax (named tensors) within Levanter for simplified management of complex sharding strategies like FSDP and Tensor Parallelism. 6. Integrating Google Cloud TPU Multislice and Ray for resilient, cost-effective compute cluster management using preemptible instances. 7. Emphasizing JAX's reproducibility guarantees (deterministic PRNGs) and using Tensorstore for deterministic data loading, enabling bit-for-bit reproducibility across training migrations and preemptions. The post also describes the Llama-style transformer architecture of Marin-8B and its adaptive training process ('Tootsie' process) across different TPU configurations.

6/24/2025

Introduces KerasHub's capability to load and utilize model checkpoints from Hugging Face Hub across different ML frameworks (JAX, PyTorch, TensorFlow). Details the integration with SafeTensors format and provides examples for loading Mistral, Llama 3.1, and Gemma models using specific KerasHub model classes (MistralCausalLM, Llama3CausalLM, Gemma3CausalLM). Highlights the flexibility of mixing model architectures with weights from various sources.