Full-Text Search is Now Generally Available In Pinecone Database

This post announces the general availability of full-text search in Pinecone, integrating BM25 keyword ranking and text-match filters directly into the vector database. It highlights the ability to handle queries for identifiers (SKUs, error codes) and exact phrases that semantic search struggles with. The implementation leverages the Tantivy library and maps onto Pinecone's existing slab architecture, allowing for a single index and schema for text, dense, and sparse vectors. Key features include Lucene query syntax, fuzzy matching, multi-language tokenization/stemming, and text-match filters for hybrid search. This addresses the business cost of silent failures and high ownership costs associated with separate lexical search systems.