
11/1/2014 · Jay Kreps
What this post added
This post announces the formation of Confluent, a company dedicated to Apache Kafka. It details the origin of Kafka at LinkedIn, driven by the need for scalable distributed data systems and real-time data processing. The post explains Kafka's core concepts as a horizontally scalable messaging system that acts as a replacement for traditional brokers, a data synchronization tool, and a foundation for real-time analytics. It highlights Kafka's design for durability, persistence, and scalability, and its evolution into a central nervous system for companies. The post also touches upon the limitations of existing open-source Kafka and the need for surrounding capabilities to build complete real-time data pipelines, which Confluent aims to address.