How Milvus Implements Dynamic Data Update And Query

This post details Milvus's internal mechanism for handling dynamic data updates and near real-time queries. It introduces the MemTable and MemTableFile architecture for managing data in memory, dividing MemTableFiles into mutable and immutable states. Data is inserted into a mutable MemTableFile, which transitions to an immutable state and is then serialized to disk by a background thread. This process ensures data is available for searching shortly after insertion, with a maximum latency of one second. The post also touches upon merging serialized files and asynchronous index building.