Unifying Stream Processing and Interactive Queries in Apache Kafka

What this post added

Introduces the 'Interactive Queries' feature for Kafka Streams. This feature allows direct querying of the state maintained by a stream processing application, treating it as an embedded, highly available, and fault-tolerant database. It contrasts this with traditional materialized views and the inefficiencies of Lambda architectures, advocating for a converged processing and storage layer within Kafka. The post details the motivation, an example of real-time risk management, and the underlying mechanism using embedded databases like RocksDB, highlighting trade-offs for using this feature versus external databases.