TechPrufer
Get started
Milvus
Milvus logoMilvus · Feature trail

Query Task Scheduling

1 post · Mar 2020

What this trail covers

Milvus has evolved its query task scheduling mechanisms to optimize performance across diverse hardware environments. Initially, a Round Robin strategy was used for multi-GPU scheduling, but this proved suboptimal on hardware with varying GPU transmission speeds. The system was updated to incorporate estimated completion times, considering data loading and computation durations for each device, to make more informed scheduling decisions. Future work includes optimizing for more complex hardware like NUMA architectures and exploring query merging for identical table queries to further enhance performance.

All Milvus trails
  1. How Does Milvus Schedule Query TasksThis post details the initial implementation of Milvus's query task scheduling. It introduces core concepts like Data Blocks (TableFiles) and Task Queues (TaskTables). The scheduling process is described, including how Milvus receives requests, creates SearchTasks for each TableFile, and selects a computing device based on estimated completion time. Performance optimizations like LRU caching and overlapping data loading with computation are discussed. A key problem identified is the performance degradation on multi-GPU hosts with different transmission speeds, which was addressed by incorporating copy time into the estimated completion time calculation. Future work on hardware complexity and query optimization is also outlined.