Introducing Gemma 4 12B: a unified, encoder-free multimodal model
Jun 3, 2026Olivier Lacombe, Gus Martins

Summary
Introduces Gemma 4 12B, a unified, encoder-free multimodal model. Key technical contributions include: replacing traditional multimodal encoders with a lightweight embedding module for vision and projecting raw audio signals directly into the LLM's dimensional space. This architecture reduces latency and memory usage, enabling local execution on consumer laptops. It also highlights advanced reasoning capabilities and the integration of Multi-Token Prediction (MTP) drafters.