KIP-848: A New Consumer Rebalance Protocol for Apache Kafka® 4.0

What this post added

Introduces KIP-848, a new, optional consumer rebalance protocol for Apache Kafka 4.0+. This protocol shifts coordination from clients to the broker-side group coordinator, enabling server-driven, incremental reconciliation. Key changes include a shift from stop-the-world rebalances to incremental partition assignment/revocation via heartbeats, server-side assignors, and dynamic server-side configuration of session and heartbeat intervals. This results in reduced consumer downtime, faster rebalances, improved stability, and simplified management. The post details the protocol's mechanics, benefits, and adoption steps, including support for live upgrades.