Log Compaction | Highlights in the Kafka and Stream Processing Community | January 2016 | Confluent

This post details the release of Kafka 0.9 and Confluent Platform 2.0. It highlights KIP-40, which adds protocol support for managing consumer groups, enabling clients to list groups and their members/lag without direct ZooKeeper interaction. It also discusses KIP-32, which aims to add a timestamp field to Kafka messages for physical time tracking. The post also introduces Kafka Connect, a new feature in 0.9 for integrating Kafka with external data systems, and provides a tutorial on building an ETL pipeline using Kafka Connect. It mentions a benchmark comparing stream processing frameworks and Microsoft's use of Kafka in Bing. The post also announces the upcoming Kafka Summit and the availability of 'Kafka, The Definitive Guide'.