Kafka Infinite Storage
Log Compaction - Highlights in the Apache Kafka ®  and Stream Processing Community - March 2017 | Confluent

Log Compaction - Highlights in the Apache Kafka ® and Stream Processing Community - March 2017 | Confluent

3/9/2017 · Gwen Shapira

What this post added

This post highlights KIP-107, which adds a `purgeDataBefore()` API in AdminClient, allowing developers to request data purging from Kafka beyond time-based and size-based cleanup policies. This is useful for multi-step stream processing jobs to remove intermediate data after it has been processed by downstream jobs.

Read the original post ↗