AI Data Storage Engine
Data Insertion and Data Persistence in a Vector Database

Data Insertion and Data Persistence in a Vector Database

4/6/2022 · Bingyi Sun

What this post added

This post details the data insertion workflow in Milvus, explaining the roles of proxy, data coord, data node, and root coord. It elaborates on data organization (collection, partition, shard/channel, segment) and the different states of segments (growing, sealed, flushed). It also describes data persistence through binlog files and the mechanisms for segment allocation and channel assignment (consistent hashing, load balancing). The Time Tick mechanism for timestamping and data consumption is also explained.

Read the original post ↗