Easily Migrate Data to the Cloud with Confluent and Databricks Connector
Nov 23, 2021Paul Earsy

Summary
Introduces the Databricks Delta Lake Sink Connector for Confluent Cloud, enabling real-time data offloading from various sources (e.g., Oracle, Salesforce) to Databricks Delta Lake. The connector supports exactly-once semantics (EOS) by staging data in S3 before committing to Delta Lake. It details the configuration steps, including setting up the target Delta Lake table with a 'partition' INT field for EOS, and configuring connector details like host name, HTTP path, token, and S3 staging bucket credentials. The post highlights the benefits of reducing operational burden, accelerating migrations, and enabling real-time analytics.