Introducing Gemma models in Keras- Google Developers Blog

What this post added

This post announces the integration of Gemma models into the KerasNLP collection, leveraging Keras 3's multi-backend support (JAX, PyTorch, TensorFlow). It introduces a new LoRA API for parameter-efficient fine-tuning, significantly reducing trainable parameters. Additionally, it details model-parallel training capabilities using Keras's distribution API, enabling distributed training across multiple GPUs/TPUs by sharding model weights. Code examples are provided for getting started, LoRA fine-tuning, and distributed training.