Kafka Connect Sink for PostgreSQL from JustOne Database | Confluent
Jun 1, 2016Duncan Pauly

Summary
This post introduces a new open-source Kafka sink connector for PostgreSQL-compatible databases. It details the use case of creating a relational data lake by streaming enterprise messages into a PostgreSQL database for agile analytics. Performance benchmarks show high ingestion rates (over 200,000 messages/second) into unindexed tables. The post explains the message-to-row conversion mechanism using JSON parsing paths and properties like `db.columns` and `db.json.parse`. It also covers database connection properties, target table definition, and delivery semantics (Fastest, Guaranteed, Synchronized), with Synchronized being the default for exactly-once delivery. Buffer size configuration and Java heap options are also discussed.