TechPrufer
Get started
Milvus
Milvus logoMilvus · Feature trail

Force Merge Compaction

1 post · Feb 2022 – Jun 2026

What this trail covers

Milvus introduces Force Merge Compaction, an asynchronous operation that consolidates small, sealed segments into fewer, larger ones. This is particularly beneficial for static, read-heavy collections where a high segment count leads to increased query fan-out overhead. By reducing the number of segments, Force Merge can significantly improve search QPS and reduce latency. The operation is triggered via the `compact()` API with a `target_size` parameter, allowing operators to control the consolidation towards a target size, potentially exceeding the per-segment cap. Experiments show significant QPS gains (76-87%) and latency reduction (p99 down ~33%) on static datasets. It is recommended for quiet windows and when write activity has ceased.

All Milvus trails
  1. How Force Merge Compaction Nearly Doubled Milvus Search QPSThis post details the experimental validation of Milvus's Force Merge Compaction feature. It quantifies the performance gains, showing an increase in search QPS from ~3,000 to ~5,600-6,000 (76-87% improvement) and a reduction in p99 latency by approximately one-third. The post explains the underlying mechanism of fan-out overhead caused by numerous small segments and how Force Merge addresses this by consolidating them into fewer, larger segments. It also outlines the experimental setup, including the dataset, hardware, and benchmark tool used, and provides guidance on when to use Force Merge (static, read-heavy collections, isolated resources, quiet windows) and when to avoid it (steady writes, single-node tight resources, peak traffic).
  2. How to Compact Data in Milvus?This post details the concept and implementation of compaction in Milvus 2.0. It explains that compaction merges small segments into larger ones and cleans up logically deleted data, reducing disk space usage and improving query efficiency. It covers binlog compaction (merging insert and delta binlogs) and segment compaction (merging sealed segments into larger ones). Trigger conditions for automatic compaction are provided for both binlog and segment compaction. Configuration parameters `dataCoord.enableCompaction` and `common.retentionDuration` are explained, along with manual triggering of compaction via `collection.compact()` and checking status with `collection.get_compaction_state()`.