Transferring Avro Data Across Schema Registries with Kafka Connect

What this post added

Introduces a custom Kafka Connect Single Message Transform (SMT) that enables the transfer of Avro data across independent Schema Registry clusters. The SMT looks up Avro schema text from a source Schema Registry, registers it in a destination Schema Registry, and updates the schema ID within the Kafka record. It also discusses potential ordering issues with schema registration and provides a workaround by setting compatibility to FULL or NONE. A demo architecture and order of operations are detailed, along with prerequisites for a Docker-based runbook.