How Small Can Google's New EmbeddingGemma 2 Get? - Qdrant

This post details experiments with Google's EmbeddingGemma 2 model, focusing on memory optimization techniques within Qdrant. It quantifies the memory savings achievable through vector quantization (1-bit TurboQuant) and dimension reduction (Matryoshka Representation Learning), comparing different configurations with and without rescoring. Key findings include achieving 30x less vector RAM with 99% quality retention using 1-bit TurboQuant without rescoring, and 77x less vector RAM with 94.5% quality retention using 256 dimensions with rescoring. The post provides concrete metrics on retained nDCG@10 and vector RAM usage for various configurations, offering practical guidance on balancing memory footprint and retrieval quality.