TechPrufer
Get started
Milvus
Milvus logoMilvus · Feature trail

Real-time Query

4 posts · Mar 2020 – Mar 2025

What this trail covers

Milvus 2.0 architecture enables real-time data query by orchestrating data loading, management, and query execution across various components. Data is loaded to query nodes from both streaming (log broker) and historical (object storage) sources. Query nodes manage historical data in sealed segments and incremental data in growing segments, with mechanisms for load balancing and failover. Incremental data is filtered by partition using flowgraphs. Real-time queries are processed via query messages, which include service_ts, travel_ts, and guarantee_ts for precise temporal querying. Query results undergo local and global reduction to ensure accuracy and deduplication.

All Milvus trails
  1. A Day in the Life of a Milvus DatumThis post details the end-to-end data flow for `_insert()` and `_search()` operations in Milvus, explaining the roles of proxy nodes, message queues (Pulsar/Kafka), data nodes, growing segments, chunks, sealed segments, index nodes, and query nodes. It describes how data is persisted to object storage, how segments are sealed, merged, and compacted, and how indexes are built and loaded for search. The post highlights the distributed design for parallelism and independent scaling of components.
  2. Milvus 2.2.12: Easier Access, Faster Vector Search Speeds, and Better User ExperienceMilvus 2.2.12 introduces the ability to retrieve raw vectors during ANN searches by setting the vector field as an output field and querying in HNSW-, DiskANN-, or IVF-FLAT-indexed collections. This eliminates the need for a two-step process (ANN search followed by ID-based query) and significantly improves vector retrieval speed.
  3. How Does the Database Understand and Execute Your Query?This post details the internal mechanisms of Milvus's query execution for scalar filtering. It explains the EBNF syntax for query expressions, the use of ANTLR and the Expr package for Plan AST generation, and the visitor design pattern for query execution, culminating in a bitmask output. It provides a concrete example of processing a complex film query.
  4. Using the Milvus Vector Database for Real-Time QueryThis post details the real-time query mechanism in Milvus 2.0. It explains how data is loaded to query nodes from log brokers (streaming) and object storage (historical), managed in sealed and growing segments, and how load balancing and failover are handled. It introduces the concept of flowgraphs for partition filtering of incremental data and the role of timeticks and tsafe values. The query message structure (msgID, collectionID, execPlan, service_ts, travel_ts, guarantee_ts) and the real-time query process, including handling of unsolved messages and the local/global reduce steps for query results, are elaborated.
  5. How Is Data Processed in a Vector Database?This post provides a deep dive into how data is processed within Milvus, elaborating on the MsgStream interface for data production and consumption, the flowgraph mechanism for filtering data by collection in shared channels, and the distinct paths for writing and reading data. It also covers the handling of DDL operations and the process of building indexes, offering a comprehensive view of Milvus's internal data handling mechanisms.
  7. Milvus 2.0 - A Glimpse at New FeaturesThis post introduces entity deletion in Milvus 2.0, explaining its asynchronous and logical nature. It details how deleted entities are marked and become invisible based on the consistency level. The post also elaborates on the four consistency levels (STRONG, EVENTUALLY, BOUNDED, SESSION) and demonstrates the performance difference between STRONG and BOUNDED consistency through latency measurements. Additionally, it introduces the 'handoff' feature, which automatically loads indexed segments to replace streaming data, enhancing search performance.
  8. How Does Milvus Schedule Query TasksThis post details the query task scheduling mechanism in Milvus, explaining how query tasks are created for each data block (TableFile), assigned to computing devices (CPU/GPU) based on estimated completion time, and processed through task queues. It introduces concepts like Data blocks (TableFile) and Task queues (TaskTable). Performance optimizations discussed include LRU caching for data blocks and overlapping data loading with computation using pipelining. It also highlights a problem with Round Robin scheduling on multi-GPU setups with different data copy speeds and the solution of incorporating copy time into device selection. Future work includes handling more complex hardware environments and query optimization through merging queries.