Gemma explained: EmbeddingGemma Architecture and Recipe- Google Developers Blog

This post details the architecture and training recipe of EmbeddingGemma, a new text embedding model derived from Gemma 3. It explains how text is converted into embeddings using a transformer, pooling, and dense layers, and describes the three weighted loss functions (NCE, GOR, Geometric Embedding Distillation) used for training. It also introduces Matryoshka Representation Learning (MRL) for flexible embedding dimensionality and outlines the development recipe including encoder-decoder training, pre fine-tuning, fine-tuning, model souping, and Quantization-Aware Training (QAT).