DeepMind

Introducing Gemma 4 12B: a unified, encoder-free multimodal model

Jun 3, 2026Olivier Lacombe, Gus Martins
Introducing Gemma 4 12B: a unified, encoder-free multimodal model
SummaryAI, from the source post

Introduces Gemma 4 12B, a unified, encoder-free multimodal model. Key technical contributions include: replacing traditional multimodal encoders with a lightweight embedding module for vision and projecting raw audio signals directly into the LLM's dimensional space. This architecture reduces latency and memory usage, enabling local execution on consumer laptops. It also highlights advanced reasoning capabilities and the integration of Multi-Token Prediction (MTP) drafters.

More from the Gemini Flash Model Series trail