Elasticsearch is Dead, Long Live Lexical Search

This post introduces Sparse-BM25 as a native lexical search capability in Milvus 2.5, building upon previous hybrid search features. It details the implementation using Tantivy for tokenization, distributed vocabulary management, and TF-IDF for sparse vector generation. The post highlights advantages over Elasticsearch, including algorithm flexibility (e.g., TW-BERT), cost efficiency through combined inverted index and dense embedding compression, and optimized long query performance using graph indices. It positions Milvus as a unified solution for both lexical and semantic search in RAG applications.