How We Built a Web-Scale Vector Database for Our Neural Network Search Engine

What this post added

This post details the technical implementation of Exa's web-scale vector database. Key contributions include: 1. Matryoshka embeddings to reduce vector dimensionality by 20x. 2. Binary quantization to further reduce memory by 16x. 3. A novel hybrid search approach using uncompressed query embeddings with binary document embeddings and a hyper-optimized dot product calculation via precomputed lookup tables, reducing computations by 1/4. 4. Loading lookup tables into CPU registers for faster access. 5. Clustering documents into 100,000 groups for targeted searching, yielding a ~1000x throughput improvement. The post also describes the integration of metadata filtering using inverted indexes and reranking with uncompressed data to maintain recall.