TechPrufer
Get started
PlanetScale
PlanetScale logoPlanetScale · Feature trail

Vector Indexing for Relational Databases

5 posts · Oct 2023 – Oct 2025

What this trail covers

PlanetScale offers GA support for vector search and storage within MySQL, making vectors a first-class part of the RDBMS. This includes relational indexing for vectors, allowing them to be stored in columns, indexed with `ALTER` or `CREATE VECTOR INDEX` statements, and queried with standard SQL `JOIN` and `WHERE` clauses within transactions. All PlanetScale features, such as branching, deploy requests, and backups, are supported. The system offers multiple distance metrics (Euclidean, inner product, cosine), supports up to 16,383 dimensions, and includes fixed and product quantization. A key innovation is the integration of SPANN (Space-Partitioned Approximate Nearest Neighbors), enabling indexes larger than RAM by combining partition-based and tree-based approaches. This allows efficient operation with vector data up to 6x bigger than available memory, with performance benefits amplified on PlanetScale Metal.

All PlanetScale trails
  1. Larger than RAM Vector Indexes for Relational DatabasesThis post details the challenges of implementing vector indexes in relational databases, particularly supporting indexes larger than RAM and maintaining transactional guarantees during inserts, updates, and deletes. It introduces the Hierarchical Navigable Small Worlds (HNSW) data structure and discusses its limitations in a relational context. The PlanetScale implementation offers both fully in-memory HNSW indexes with transactional guarantees and a new hybrid vector index design for indexes larger than RAM, ensuring approximate nearest neighbor search behaves as expected within a relational database.
  2. PlanetScale vectors is now GAThis post announces the General Availability (GA) of PlanetScale's vector search and storage capabilities for MySQL. It details improvements made since the public beta, including doubled query performance and an eightfold increase in memory efficiency. The core contribution is the integration of vectors as a first-class relational index within MySQL, supporting standard SQL operations, transactions, and all existing PlanetScale features like branching and backups. It highlights advanced index features such as support for Euclidean, inner product, and cosine distances, up to 16,383 dimensions, and fixed/product quantization. A significant technical advancement is the incorporation of SPANN (Space-Partitioned Approximate Nearest Neighbors) into an RDBMS, enabling indexes larger than RAM by using a hybrid approach that stores vectors in hidden InnoDB tables and a portion in memory for efficient partition identification. The post also provides instructions on how to enable vector support and links to further resources.
  3. Announcing the PlanetScale vectors public betaThis post announces the public beta of PlanetScale's vector search and storage capabilities, which allow storing vector data directly within MySQL. It details the underlying architecture, which is based on Microsoft Research's SPANN and SPFresh algorithms, extended with transactional support and integration into InnoDB. The post contrasts this approach with HNSW and DiskANN, highlighting the benefits of incremental updates, ACID compliance, and scalability to terabytes. It also explains how to enable the beta feature at the branch level.
  4. PlanetScale is bringing vector search and storage to MySQLThis post announces the upcoming addition of vector storage and search capabilities to PlanetScale's MySQL offering. It explains the concept of vectors and their application in machine learning through embeddings. The technical details highlight the implementation of the Hierarchical Navigable Small World (HNSW) algorithm for efficient vector similarity search and the introduction of a first-class vector data type within MySQL. The post also mentions that PlanetScale's existing MySQL fork will be updated to include these features, and packages will be available for local development.