Real-time Field Service Management Optimization

8 posts · Jul 2020 – Jan 2024

What this trail covers

This feature thread tracks the evolution of real-time order notifications and logistics management. It details how companies have migrated from siloed, batch-processed data systems to modern microservices architectures powered by Confluent's event streaming platform. Key technical aspects include integrating disparate databases (Postgres, MySQL, SQL Server) using CDC connectors, leveraging ksqlDB for real-time data processing and microservice development, and utilizing Stream Governance for data.

All Confluent trails

Jan 2024How To Build a Real-Time Field Ops Alerting SystemThis post details Confluent's internal use of Confluent Cloud to build a real-time alerting system for its Field Operations teams. It describes the challenges of translating events into actionable outcomes, the architecture for capturing and enriching events using Confluent connectors, Cluster Linking, and Kafka Streams, and delivering alerts via Slack. The post provides code snippets for producing events from BigQuery and for Kafka Streams DSL to enrich connector failure events with account team and Slack user information.
Dec 2023Real-Time Field Service Management OptimizationThis post details a telco company's implementation of a real-time field service management system using Confluent Cloud. The solution involves modifying the source CRM to generate real-time events, feeding them into a "Topic-as-a-Service" architecture and an API gateway. HTTP sink connectors are used to route events to third-party service providers, enabling immediate updates on service task status. The architecture leverages Confluent Cloud, HTTP sink connectors, stream processing (Flink), Stream Governance (Schema Registry), and Terraform for provisioning. The business outcomes include incident-free operations, improved collaboration, enhanced efficiency, scalability, and increased revenue.
Oct 2023Real-Time Inventory in Retail with ConfluentThis post details building a real-time inventory system for retail using Confluent Cloud. It covers consuming data from various sources (in-store and e-commerce databases) using CDC connectors, processing this data with ksqlDB to create streams for sales and receipts, and then aggregating these into an 'inventory' stream. A ksqlDB table 'stock_on_hand' is constructed to provide a real-time, queryable view of inventory levels per SKU. The post also discusses distributing this real-time inventory data to systems like MongoDB for enhanced fulfillment decisions and customer-facing applications, highlighting the use of over 120 pre-built connectors and stream processing tools like Kafka Streams and Flink.
Jul 2023Acertus Real Time Order NotificationsThis post details ACERTUS's implementation of real-time order notifications by migrating from siloed, batch-processed data systems to a microservices architecture using Confluent's event streaming platform. It highlights the technical challenges of integrating multiple databases (Postgres, MySQL, SQL Server) and the adoption of Confluent's CDC connectors, ksqlDB for stream processing and microservice development, and Stream Governance features (Schema Registry, Stream Lineage). The post emphasizes Confluent's advantages over alternatives like Kinesis and MSK, particularly its infinite storage, managed connectors, and ksqlDB capabilities, enabling ACERTUS to provide instant customer updates and improve operational efficiency.
Jun 2023How Mobile Premier League Reduced Player Churn with Confluent CloudMobile Premier League (MPL) migrated from a batch pipeline system to Confluent Cloud to enable real-time data processing for improved player matchmaking, faster decision-making, and real-time fraud detection. They process hundreds of millions of events daily, including client and server events from their gaming app, relational databases, and marketing SaaS apps. This enables personalized in-app offers, customized onboarding journeys, and real-time anomaly detection for fraud prevention, leading to reduced player churn and increased operational efficiency.
Advertisement
Apr 2021Data Streams Are Nothing Without Actionable Insights Leading to Real-Life ActionsThis post details the development of a custom stateful connector that bridges Apache Kafka event streams with the Camunda workflow engine for a vehicle maintenance use case. It addresses challenges in translating raw sensor data into actionable insights by implementing solutions for duplicate event prevention, time windowing (using ksqlDB or Kafka Streams), and semantic aggregation. The custom connector, built with Java and Kafka/Camunda clients, maintains a persistent history to enable semantic aggregation and stream replay, facilitating the triggering of workflow instances (e.g., scheduling maintenance) or cancellation based on event streams.
Nov 2020Digital Transformation in Style: How Boden Modernizes Retail Using KafkaThis post details Boden's digital transformation journey using Confluent Cloud and Apache Kafka. It highlights the migration from a legacy monolith to a microservices architecture to address integration challenges and the need for near real-time data. Key technical implementations include using Kafka for exposing product data changes from a PIM system, avoiding point-to-point integrations, and leveraging Schema Registry for event schema versioning. The post also describes using ksqlDB to aggregate clickstream data and feed it into Snowflake for business analytics, enabling insights into customer journeys and buying preferences. It also touches upon establishing standards for business events and architecture patterns for domain services, including aggregation services.
Jul 2020Implement CDC & Streaming Analytics Using Kafka & DebeziumThis post details the implementation of Change Data Capture (CDC) using Debezium and Kafka Streams to ingest data from MongoDB into Kafka. It addresses the challenge of handling partial updates from the Debezium MongoDB CDC Connector by using Kafka Streams to maintain a table of complete documents. The post explains the concepts of streams and tables in Kafka Streams, provides a Java code example for a Kafka Streams application that performs a left join between a CDC stream and a table to merge changes, and details the merge function logic for handling create, read, update, and delete operations.