How Bolt Adopted Change Data Capture with Confluent for Real-Time Data & Analytics
Aug 20, 2020Ruslan Gibaiev
Summary
This post details Bolt's adoption of Change Data Capture (CDC) with Confluent Platform for real-time data and analytics. It explains the shortcomings of their previous batch data replication methods and outlines the requirements for a new pipeline, including linear scalability and proprietary data transformations. The article highlights the benefits of log-based CDC, the role of Debezium for capturing data changes, Apache Kafka as the core event streaming platform for ordered and replayable data, Kafka Connect for orchestrating CDC processes, Confluent Schema Registry for schema management and data contracts, and Kafka Streams/ksqlDB for stream processing and data transformation.
More from the Real-time Insurance Quoting with Data Streaming trail
- Accelerating Insurance Claims With a Stream Processing Architecture
- Transforming T+1 Trade Settlement for Modern Markets With Confluent
- Data Streaming in Healthcare: Achieving the Single Patient View
- Real-Time Order Management: The Key to Streamlining Your Business Operations
- Real-Time Inventory in Retail with Confluent