Gemma 3 QAT Models: Bringing state-of-the-Art AI to consumer GPUs- Google Developers Blog

This post introduces new versions of Gemma 3 models optimized with Quantization-Aware Training (QAT). QAT reduces memory requirements by quantizing model parameters to fewer bits (e.g., int4) while maintaining high quality by incorporating the quantization process during training. This enables running powerful models like Gemma 3 27B locally on consumer-grade GPUs (e.g., NVIDIA RTX 3090) by dramatically reducing VRAM requirements (e.g., Gemma 3 27B drops from 54 GB to 14.1 GB). The post also details integration with popular developer tools like Ollama, LM Studio, MLX, Gemma.cpp, and llama.cpp.