Milvus

Full-Text Search in Milvus - What's Under the Hood

Apr 15, 2025Buqian Zheng
Full-Text Search in Milvus - What's Under the Hood
SummaryAI, from the source post

Introduces and details the implementation of Sparse-BM25 full-text search in Milvus 2.5. Explains the BM25 algorithm and its transformation into an Inner Product calculation for sparse vectors. Details Milvus's novel approach using dynamic statistics (avgdl, term distribution) and search-time encoding to handle the complexities of BM25 computation internally, removing client-side burden. Provides a PyMilvus example for defining a collection with a text field and a BM25 function, creating an AUTO_INDEX with BM25 metric, inserting raw text, and performing searches.

More from the AI Data Storage Engine trail