Apache Kafka Supports 200K Partitions Per Cluster | Confluent

This post details significant improvements in Apache Kafka 1.1.0 that increase the number of partitions a cluster can support. Key enhancements include optimizing the controller's controlled shutdown process by using asynchronous ZooKeeper writes and batching leader change communication, reducing shutdown time from 6.5 minutes to 3 seconds. Controller failover time was also improved by using asynchronous ZooKeeper APIs for state reloading, resulting in a 100% improvement in reload time. These changes enable Kafka clusters to support up to 200,000 partitions.