Kafka Client Migration Automation

1 post · Feb 2026 – Apr 2026

What this trail covers

Confluent Cloud Gateway now fully orchestrates Kafka client migrations with KCP, enabling seamless, safe, and rollback-friendly cutovers. This includes transparent authentication translation (e.g., SASL/SCRAM to API keys), group-based migration for incremental rollouts, and offset preservation via Cluster Linking, minimizing downtime and coordination overhead. The process is streamlined into three KCP commands: init, migrate, and promote. This post demonstrates a POC for disaster recovery failover orchestration using Confluent Cloud Gateway to achieve seamless client failover within 60 seconds, combining Cluster Linking for server-side replication with the gateway for client-side redirection and managing the replication reversal.

All Confluent trails

Apr 2026Kafka Client Migrations With KCP and Confluent Cloud GatewayThis post details the integration of Confluent Cloud Gateway with KCP for end-to-end Kafka client migration automation. It introduces the concept of streaming domains and routes in Gateway for proxying traffic and transparently translating authentication mechanisms (e.g., SASL/SCRAM to API keys). The migration strategy is broken down into four stages: Deploy Gateway, Plan Migration Groups, Onboard Clients, and Migrate Group by Group. The core client migration is executed via three KCP commands: `kcp migration init` for validation and planning, `kcp migration migrate` for the cutover, and `kcp migration promote` for finalization. The post highlights the benefits of Gateway's authentication swapping and group-based migration for reducing coordination and risk.
Feb 2026Automate Kafka Migration With Kafka Copy Paste (KCP)This post introduces KCP (Kafka Copy Paste), an open-source tool by Confluent that orchestrates the entire Kafka migration workflow to Confluent Cloud. KCP automates discovery (inventorying clusters, topics, ACLs, schemas, connectors, clients), provisioning of target Confluent Cloud infrastructure (including networking like PrivateLink via Terraform), and data migration using Cluster Linking. It currently supports migrations from Amazon MSK to Confluent Cloud, with self-managed Kafka support coming soon. Future client migration will be a single command leveraging Confluent Cloud Gateway for seamless cutovers.
Feb 2026Building Kafka Client Failover: A 60-Second disaster recovery POC with Confluent Cloud GatewayThis post details a Proof of Concept (POC) for achieving near-instantaneous disaster recovery failover for Kafka clients using Confluent Cloud Gateway. It highlights the challenges of client-side failover due to static bootstrap server configurations and the need for a proxy to redirect traffic. The POC demonstrates how Confluent Cloud Gateway can intercept Kafka client traffic and redirect it to a passive cluster during an outage. It also addresses the complexity of reversing Cluster Linking replication and managing read-only mirror topics during failover, proposing a workflow that includes dropping in-flight connections, reversing replication, switching the gateway, and accepting new connections. The demo showcases identity passthrough and OAuth2 support for client credentials.
Advertisement