Blogs›Milvus›Data File Cleanup Mechanism
Milvus has evolved its data file cleanup mechanism. Previously, files marked for soft-delete were hard-deleted after a fixed 5-minute interval. This approach was unreliable, potentially causing query failures if queries exceeded this duration, and led to excessive disk usage due to delayed deletion of combined files. The improved strategy in v0.6.0 uses a reference counting mechanism. Files are hard-deleted only when they are no longer in use by any active task and have been soft-deleted. This ensures timely removal of unused files, reducing disk footprint and improving system reliability.
12/18/2019
Introduced a new data file cleanup strategy in Milvus v0.6.0. The previous 5-minute soft-delete to hard-delete interval was replaced with a reference counting system. Files are now hard-deleted only when they are not in use by any task and have been soft-deleted, preventing query failures and reducing disk usage by ensuring timely removal of unused files.