How to Contribute a Kafka Connector on Confluent Hub

What this post added

This post details the process of developing and contributing a custom Kafka connector to Confluent Hub, using the OpenSky Network Source Connector as a practical example. It elaborates on key aspects of connector development such as task parallelism for scalability, connector naming conventions, schema modeling with considerations for data translation and naming, the mechanism for committing offsets to ensure fault tolerance, and the importance of unit testing. Additionally, it outlines the steps for distributing the connector via Confluent Hub, including licensing considerations and the process for vendor verification. Finally, it demonstrates a complete workflow of a connector in action, integrating with ksqlDB and Elasticsearch for data visualization.