Raft or not? The Best Solution to Data Consistency in Cloud-native Databases

What this post added

This post analyzes the limitations of consensus-based replication (Raft, Paxos) in cloud-native databases, particularly concerning availability, complexity, and performance. It proposes an alternative log replication strategy for cloud-native and distributed databases based on two principles: 1) Replication as a service, decoupling synchronization from storage, and 2) The 'Russian doll principle', leveraging external services for logging (e.g., Kafka/Kinesis) and storage (e.g., S3), and using coordination services (e.g., etcd) for leader election. The post also touches upon Milvus's tunable consistency features, such as follower reads, as a means to improve cluster throughput and reduce leader load.