Predicting Flight Arrivals with the Apache Kafka Streams API

What this post added

This post demonstrates how to use the Kafka Streams API, specifically KTables and GlobalKTables, to perform real-time machine learning predictions and online model retraining. It details the architecture for consuming flight data, joining it with a GlobalKTable holding serialized machine learning models (OnlineLogisticRegression), and using these to predict flight arrival status. It also explains how to update these models in real-time by publishing new serialized models to a Kafka topic that feeds the GlobalKTable. The post highlights the benefits of this approach for continuous model improvement without offline batch training.