Introducing Milvus 2.5: Full-Text Search, More Powerful Metadata Filtering, and Usability Improvements!

What this post added

Milvus 2.5 introduces direct full-text search (FTS) capabilities, also known as lexical or keyword search, using the BM25 metric based on sparse vectors. This allows for inserting and querying text directly, with Milvus handling vector conversions internally. The implementation integrates with the tantivy ecosystem for tokenization and supports raw text ingestion and querying. It also introduces Sparse-BM25 for relevance scoring and optimizations like graph indexing and vector quantization. Additionally, a 'Text Match' feature is added for filtering based on specific text strings, built on tokenization and inverted indexes. Scalar filtering performance is significantly enhanced with the introduction of BitMap Index and Array Inverted Index, improving performance for tag filtering and other metadata filtering scenarios.