TechPrufer
Get started
Qdrant
Qdrant logoQdrant · Feature trail

Data Structure Optimization

1 post · Aug 2024 – Aug 2026

What this trail covers

Qdrant continues to enhance its core data structures and indexing capabilities to improve memory usage, storage efficiency, and query performance, particularly for multitenant workloads and large datasets. New features include defragmentation of tenant storage, on-disk support for payload indexes, and a more memory-efficient UUID datatype for payload indexes. The Query API has been expanded with a Groups endpoint for grouping results by payload fields, random sampling, and distribution-based score fusion.

All Qdrant trails
  1. Qdrant 1.19 - TurboQuant Datatype & Memory Tiers - QdrantVersion 1.19 introduces the Turbo4 datatype, a new storage format that compresses vectors to four bits as the sole representation, reducing storage by up to nine times compared to TurboQuant quantization. It also introduces a unified `memory` parameter for configuring memory tiers (`pinned`, `cached`, `cold`) across collection components, replacing older per-component flags. Per-tenant IDF statistics are now available to narrow the corpus for IDF calculations, improving BM25 scoring in multi-tenant deployments. Filtering enhancements include prefix matching on keyword fields and a new slice filter condition for partitioning a collection's points into deterministic, disjoint subsets. The Web UI gains live resharding progress, an overhauled Collection Visualizer with server-driven computation and WebGL2 rendering, and interactive payload index configuration.
  2. Qdrant 1.12 - Distance Matrix, Facet Counting & On-Disk Indexing - QdrantThis post introduces several new features in Qdrant 1.12: the Distance Matrix API for calculating pairwise vector distances (with 'pairs' and 'offsets' output formats), the Facet API for counting unique values in payload fields (with 'exact' option for precision), and on-disk support for Text and Geo indexes. The Distance Matrix API is integrated into the Graph UI for enhanced data exploration. The on-disk indexing for text and geo data aims to reduce memory usage, with performance considerations for cold query latency and memory vs. disk trade-offs. The Facet API's default behavior is approximate filtering, with an option for exact counts.
  3. Qdrant 1.11 - The Vector Stronghold: Optimizing Data Structures for Scale and Efficiency - QdrantThis post introduces Qdrant 1.11, focusing on optimizations to data structures for improved memory usage and efficiency, particularly for multitenant workloads. Key contributions include: 1. Defragmentation of tenant storage to co-locate vectors of the same tenant for faster bulk reading. 2. On-disk support for payload indexes, allowing less frequently accessed data to be stored on disk, freeing up RAM for hot data. 3. A new UUID datatype for payload indexes that stores UUIDs as bytes, reducing memory usage by up to 2.25x compared to string storage. Additionally, the Query API is enhanced with a Groups endpoint for grouping results by payload fields, random sampling, and distribution-based score fusion.