Kafka Consumer Offsets Guide—Basic Principles, Insights & Enhancements

This post introduces KIP-1094, which adds a new constructor method with nextOffsets to ConsumerRecords, enabling more accurate and reliable retrieval of offset and leader epoch information. It also elaborates on manual offset committing strategies (commitSync, commitAsync) and highlights the critical importance of committing the leader epoch alongside the offset to ensure consumer group consistency and prevent issues like OFFSET_OUT_OF_RANGE errors during leader changes and rebalances.