Evolving Pinterest’s Embedding Retrieval Platform

This post details the evolution of Pinterest's Manas embedding retrieval platform, focusing on three key areas: Quantization (Scalar Quantization and Product Quantization) to reduce memory footprint and serving costs, with benchmarks showing trade-offs between compression and recall, and the implementation of Linear Scaling SQ with SIMD intrinsics for compute efficiency; SSD-based Serving using SPANN with PQ quantization to enable serving ANN queries directly from SSD, demonstrating significant memory and CPU savings compared to in-memory solutions; and Multi-embedding Retrieval to move beyond the two-tower model's single-vector constraint, enabling late interaction models like ColBERT with "Sum of MaxSim" scoring logic and updates to the Manas query parser to handle multiple query embeddings.