It's Okay To Store Data In Kafka

What this post added

This post argues that Kafka is suitable for long-term data storage, not just as a messaging queue. It explains use cases like event sourcing, populating caches, recomputing stream processing results, and Change Data Capture (CDC). The post highlights Kafka's design for persistence, disk storage, checksumming, and replication for fault tolerance, noting that performance does not degrade with data accumulation. It contrasts Kafka with traditional messaging systems that lack robust storage and replication, and positions Kafka as a 'Streaming Platform' rather than a message queue. The post also discusses Kafka's role as a commit log in a datacenter's 'giant database' architecture, with derived indexes and views served by other systems, and mentions Kafka Streams and KSQL for materialized views and queries.