The Simplest Useful Kafka Connect Data Pipeline in the World – Part 2

What this post added

This post details the second part of a series on building a Kafka Connect data pipeline, specifically focusing on streaming data from Kafka to Elasticsearch. It provides configuration details for the Elasticsearch sink connector, demonstrates loading the connector using the Confluent CLI, and shows how to query the data in Elasticsearch. It highlights that Kafka Connect sets the document ID in Elasticsearch to topic/partition/offset for exactly-once delivery and that the schema of the source MySQL table is used for mapping in Elasticsearch. It also mentions the use of Kibana for visualizing the data.