AI Data Storage Engine
Full-Text Search in Milvus - What's Under the Hood

Full-Text Search in Milvus - What's Under the Hood

4/15/2025 · Buqian Zheng

What this post added

Introduces and details the implementation of Sparse-BM25 full-text search in Milvus 2.5. Explains the BM25 algorithm and its transformation into an Inner Product calculation for sparse vectors. Details Milvus's novel approach using dynamic statistics (avgdl, term distribution) and search-time encoding to handle the complexities of BM25 computation internally, removing client-side burden. Provides a PyMilvus example for defining a collection with a text field and a BM25 function, creating an AUTO_INDEX with BM25 metric, inserting raw text, and performing searches.

Read the original post ↗