How Does Milvus Schedule Query Tasks

What this post added

This post details the query task scheduling mechanism in Milvus, explaining how query tasks are created for each data block (TableFile), assigned to computing devices (CPU/GPU) based on estimated completion time, and processed through task queues. It introduces concepts like Data blocks (TableFile) and Task queues (TaskTable). Performance optimizations discussed include LRU caching for data blocks and overlapping data loading with computation using pipelining. It also highlights a problem with Round Robin scheduling on multi-GPU setups with different data copy speeds and the solution of incorporating copy time into device selection. Future work includes handling more complex hardware environments and query optimization through merging queries.