Introducing Apache Kafka 3.7

What this post added

Introduces early access support for JBOD disk failure handling in KRaft (KIP-858). Adds broker-side support for client-level metrics via a standardized telemetry interface (KIP-714) and a mechanism to configure these metrics (KIP-1000). Previews the next generation of the Consumer Rebalance Protocol for improved efficiency (KIP-848). Optimizes leader discovery for clients to reduce latency during leadership changes (KIP-951). Provides an official Apache Kafka Docker image (KIP-975). Implements exponential backoff for Kafka client retries to reduce overload after broker failures (KIP-580). Adds new metrics for Tiered Storage monitoring (KIP-963). Enables independent stopping of KRaft processes (KIP-979). Adds server-side transaction verification to prevent hanging transactions (KIP-890). Marks old client protocol API versions (pre-2.1) as deprecated for removal in Kafka 4.0 (KIP-896). Allows AdminClient to communicate directly with the KRaft Controller Quorum for operations like DESCRIBE_QUORUM and INCREMENTAL_ALTER_CONFIGS (KIP-919). Enables dynamic reloading of certificates with different DN/SANs (KIP-978). Adds a `CurrentControllerId` metric to identify the current controller (KIP-1001). Introduces 11 new metrics for measuring KRaft performance (KIP-938). Deprecates Java 11 support for Kafka broker and tools, planned for removal in Kafka 4.0 (KIP-1013). In Kafka Streams, introduces rack-aware task assignment strategies (`balanced_subtopology`) (KIP-925). Expands default DSL store configuration to custom types with support for RocksDB and in-memory stores (KIP-954). Relaxes the non-null key requirement for certain left-joins (KIP-962). Adds a new interface for handling standby task updates (KIP-988). Adds `VersionedKeyQuery` and `MultiVersionedKeyQuery` for querying versioned state stores (KIP-960, KIP-968). Adds `reverseRange` and `reverseAll` queries for key-value stores in IQv2 (KIP-985).