Embedding API Performance Benchmarking

1 post · Jun 2024 – May 2025

What this trail covers

Milvus provides a TextEmbedding Function interface that integrates popular embedding models, streamlining vector search pipelines. This feature enables direct comparison and benchmarking of various embedding API providers under realistic deployment scenarios, focusing on latency and network effects. The goal is to help developers optimize RAG systems and other AI applications by understanding the performance trade-offs of different embedding services. This post details the integration of dense and sparse embedding models, as well as rerankers, into PyMilvus, enabling local execution with Milvus Lite and demonstrating practical use cases.

All Milvus trails

May 2025We Benchmarked 20+ Embedding APIs with Milvus: 7 Insights That Will Surprise YouThis post details a benchmark of over 20 embedding APIs using Milvus's TextEmbedding Function. It highlights the significant impact of network latency on API performance, comparing results across North America and Asia. Key findings include surprising performance rankings, varying model size impacts, complex trade-offs with token length and batch size, API reliability risks, the competitiveness of local inference, and the negligible overhead introduced by Milvus itself. It provides actionable tips for optimizing RAG embedding performance, emphasizing localized testing and strategic geo-matching of providers.
Jun 2024Introducing PyMilvus Integration with Embedding ModelsThis post introduces the integration of dense and sparse embedding models, and rerankers into the PyMilvus SDK. It demonstrates how to use `DefaultEmbeddingFunction` for dense embeddings (e.g., `all-MiniLM-L6-v2`), `BM25EmbeddingFunction` for sparse embeddings (including fitting and saving parameters), and `JinaRerankFunction` for reranking search results. The examples utilize Milvus Lite for local execution, showcasing practical applications for generating embeddings and performing searches.
Advertisement