Run Ray on TPU, Part 2: Ray AI libraries- Google Developers Blog

What this post added

This post details the integration of Ray AI libraries (Ray Serve, Ray Data, and Ray Train) with Google Cloud TPUs. It explains how Ray Serve leverages TPUs for LLM inference, using a `topology` field in `accelerator_config` to ensure multi-host models are placed on a single TPU slice. Ray Data introduces `iter_jax_batches()` for efficient, device-sharded JAX array batching for training and inference. JaxTrainer is presented as a solution for distributed JAX training on TPUs, simplifying topology management and providing fault tolerance. The post also highlights the availability of official `rayproject/ray:*-tpu` Docker images and enhanced Ray Dashboard metrics for TPU utilization.