Streaming Machine Learning

This feature thread tracks the evolution of integrating machine learning with real-time data streams. It covers transactional machine learning (TML) which combines automated machine learning (AutoML) with data streams using Apache Kafka as the backbone. The focus is on creating frictionless ML processes, building transactional training datasets from streams, applying ML algorithms, and using them for predictive analytics and optimization. The architecture leverages tools like MAADS-VIPER and MAA. This post details how to build, operate, and monitor analytic models in a scalable, mission-critical way by leveraging Apache Kafka® as a central, scalable, mission-critical nervous system plus Apache Kafka's Streams API to build intelligent streaming applications. It discusses potential use cases for machine learning in mission-critical real time applications and outlines a reference architecture for building, operating and monitoring analytic models with Kafka, separating model training from model inference.