Achieving Same Day Delivery with Confluent Data Streaming

This post details a grocery delivery company's migration from a self-managed Apache Kafka on Azure HDInsight to Confluent Cloud to enable same-day delivery. The previous setup suffered from maintenance burdens, missed upgrades/patches, frequent pipeline failures, lack of managed connectors, and no stream processing. The migration to Confluent Cloud leverages multi-AZ dedicated clusters with VNET peering for high availability, pre-built managed connectors (Debezium MongoDB, SQL Server, JDBC Sink) for data ingestion and egress, and Schema Registry for data consistency. The .NET client library was used for application integration. The result is improved reliability, reduced downtime, and freed-up engineering resources for innovation.