Bottled Water: Real-time integration of PostgreSQL and Kafka | Confluent

What this post added

Introduces Bottled Water, an open-source tool for PostgreSQL to Kafka integration. Leverages PostgreSQL 9.4's logical decoding to capture row-level change events (inserts, updates, deletes) and coordinates them with a consistent snapshot. Encodes data in Avro format and streams it to Kafka, with each table becoming a topic and each row a message keyed by its primary key. Utilizes Kafka's log compaction feature to retain the latest message for each key indefinitely, effectively storing both the database snapshot and real-time changes within Kafka.