TechPrufer
Sign inGet started

Blogs›Confluent›Real-time Insurance Quoting with Data Streaming

Real-time Insurance Quoting with Data Streaming

Real-time Insurance Quoting with Data Streaming

4
posts
2018–2023

This release enhances Confluent Cloud's capabilities for real-time order notifications and logistics management. It details how ACERTUS migrated from siloed, batch-processed data systems to a modern microservices architecture powered by Confluent's event streaming platform. Key technical aspects include integrating disparate databases (Postgres, MySQL, SQL Server) using CDC connectors, leveraging ksqlDB for real-time data processing and microservice development, and utilizing Stream Governance for data quality and lineage. This post details building an end-to-end IoT motion detection and alerting system using Confluent Cloud and ksqlDB. It covers hardware choices (Raspberry Pi Zero W), software components (Motion, librdkafka, Docker), and the architecture for ingesting motion events into Kafka. The system utilizes ksqlDB for real-time stream processing and querying, enabling both real-time monitoring with push queries and historical event search with pull queries. The UI is built with React and Next.js, demonstrating a practical application of stream processing for fraud detection.

2023

Providing Real-Time Insurance Quotes via Data Streaming

9/6/2023

This post details the technical approach to enabling real-time insurance quotes by migrating data from a monolithic database to a streaming pipeline using Confluent Cloud. It highlights the use of Change Data Capture (CDC) connectors (specifically mentioning Debezium PostgreSQL Connector) to stream row-level changes to Kafka topics. The post provides a JSON configuration example for a CDC connector and discusses the benefits of this approach for overcoming scalability issues, database load, and enabling real-time event consumption by modern microservices. It also touches upon future integrations with sink connectors for platforms like Elasticsearch, Snowflake, and cloud storage, as well as potential applications in IoT, ML, and GenAI.

2022

Risk Management in Volatile Markets with Data Streaming and Analytics

7/7/2022

This post details the integration of Confluent Cloud and Databricks for real-time risk management in financial markets. It outlines an architecture that uses Confluent's streaming capabilities to ingest and enrich market and portfolio data, feeding it into Databricks' Lakehouse for advanced risk calculations. Key components include the SSE Source Connector for market data, ksqlDB for real-time enrichment and joining with portfolio data, and the Databricks Delta Lake Sink Connector for exactly-once semantics data writing. Delta Live Tables are used for incremental transformations and Monte Carlo simulations, with results visualized through Databricks SQL.

2020

Leverage Real-Time Clickstream Data Using ksqlDB and Elasticsearch

5/26/2020

This post details building an end-to-end IoT motion detection and alerting system using Confluent Cloud and ksqlDB. It covers hardware choices (Raspberry Pi Zero W), software components (Motion, librdkafka, Docker), and the architecture for ingesting motion events into Kafka. The system utilizes ksqlDB for real-time stream processing and querying, enabling both real-time monitoring with push queries and historical event search with pull queries. The UI is built with React and Next.js, demonstrating how to leverage real-time clickstream data using ksqlDB and Elasticsearch for analysis and visualization.

2018

ATM Fraud Detection with Apache Kafka and KSQL | Confluent

10/29/2018

This post details building an end-to-end IoT motion detection and alerting system using Confluent Cloud and ksqlDB. It covers hardware choices (Raspberry Pi Zero W), software components (Motion, librdkafka, Docker), and the architecture for ingesting motion events into Kafka. The system utilizes ksqlDB for real-time stream processing and querying, enabling both real-time monitoring with push queries and historical event search with pull queries. The UI is built with React and Next.js, demonstrating a practical application of stream processing for fraud detection. Specifically, it showcases how ksqlDB can be used for stream-stream joins within a time window to identify fraudulent ATM transactions based on account, location, and time proximity.