Stream Processing Fundamentals
Watermarks, Tables, Event Time, and the Dataflow Model | Confluent

Watermarks, Tables, Event Time, and the Dataflow Model | Confluent

5/3/2017 · Eno Thereska

What this post added

Critiques the Google Dataflow model's use of watermarks and triggers for handling time and out-of-order data in stream processing. Proposes Kafka Streams' 'Table' abstraction as a more general and simpler approach. Explains how Kafka Streams handles mutable data and windowed computations by treating them as continuously updated tables. Discusses operational tuning parameters (commit interval, cache size) for managing output volume and update lag. Highlights the benefits of interactive queries for accessing the latest results.

Read the original post ↗