Why ZooKeeper Was Replaced with KRaft - The Log of All Logs | Confluent Why Replace ZooKeeper with Kafka Raft - The Log of All Logs

What this post added

This post delves into the technical reasons for replacing ZooKeeper with KRaft for Kafka metadata management. It explains the scalability limitations of the old controller design with ZooKeeper, particularly during broker shutdowns and controller failovers, detailing the impact of ZooKeeper's read/write load and Znode limitations. It then introduces KRaft as a solution, explaining its implementation as a metadata log using the Raft consensus algorithm for quorum replication, and the necessity of a new leader election protocol within KRaft to overcome ZooKeeper's limitations.