TechPrufer
Get started
Milvus
Milvus logoMilvus · Feature trail

In-Memory Replicas

1 post · Aug 2022

What this trail covers

Milvus 2.1 introduces in-memory replicas to enhance read throughput and system availability. This feature allows data segments to be loaded onto multiple query nodes, enabling parallel search requests and improving resilience to query node failures. Configuration involves specifying the replica number during collection loading, with limitations based on available memory. Milvus 2.1 also introduces a new routing protocol that significantly reduces retrieval latency for small datasets, bringing it down to 5ms. It fine-tunes its concurrency model with a new cost evaluation model and concurrency scheduler, merging small-nq queries for a 3.2x performance boost in high concurrency scenarios. Data loading is improved with Zstandard compression for binary logs and concurrent segment loading using goroutine pools. Milvus 2.1 adds support for variable-length string (VARCHAR) as a scalar data type, usable as primary keys and attribute filters, and introduces a scalar inverted index using MARISA-Tries for faster filtering on strings. It also offers the option to use Kafka as a message storage alongside Pulsar, and its Java SDK is now production-ready. Observability is enhanced with new monitoring metrics and optimized log keeping. Embedded Milvus is introduced for easier local development and prototyping.

All Milvus trails
  1. Increase Your Vector Database Read Throughput with In-Memory ReplicasThis post introduces the concept and implementation of in-memory replicas in Milvus 2.1. It explains how loading a collection with `replica_number` greater than 1 allows data segments to be replicated across multiple query nodes. This enhances read throughput by enabling parallel search requests and improves system availability by providing immediate failover to another query node if one crashes. The post details the configuration process via `collection.load(replica_number=N)` and how to check replica information using `collection.get_replicas()`, noting memory limitations.
  2. What's new in Milvus 2.1 - Towards simplicity and speedMilvus 2.1 introduces in-memory replicas to enhance read throughput and system availability by allowing data segments to be loaded onto multiple query nodes for parallel search requests and improved resilience. It also introduces a new routing protocol that reduces retrieval latency for small datasets to 5ms. The concurrency model is refined with a new cost evaluation model and concurrency scheduler, merging small-nq queries for a 3.2x performance boost. Data loading is optimized with Zstandard compression for binary logs and concurrent segment loading using goroutine pools. Support for VARCHAR as a scalar data type and a scalar inverted index using MARISA-Tries for string filtering is added. Kafka is now supported as a message storage option alongside Pulsar. The Java SDK is production-ready, and observability is improved with new metrics and optimized logging. Embedded Milvus is introduced for local development.