Confluent

How to Remove Apache Kafka Brokers the Easy Way

May 3, 2022Stanislav Kozlovski
How to Remove Apache Kafka Brokers the Easy Way
SummaryAI, from the source post

Introduces the ability to safely remove multiple Kafka brokers with a single command, addressing previous limitations of removing only one broker at a time and the risk of under-replicated partitions. Details the use of Confluent's Self-Balancing Clusters for intelligent partition reassignment and bin-packing, operation persistence using a state machine and a Kafka topic for resilience, preliminary validations to prevent capacity issues, and a new Broker Replica Exclusion API to handle race conditions with new topic creations.

More from the Kafka Broker and Streams Enhancements trail