Machine Learning and AI at Scale with MAADS-VIPER and Apache Kafka

This post introduces Transactional Machine Learning (TML) as a paradigm that integrates automated machine learning (AutoML) with data streams using Apache Kafka. It details the five principles of TML: fluid data, joinable streams, standardized stream format (JSON), integration with AutoML, and low-code development. The post highlights how Kafka's offset management and Confluent Tiered Storage are crucial for creating transactional training datasets. It also presents the MAADS-VIPER connector and MAADS-HPDE AutoML technology as key components for building TML solutions, and outlines the MAADS Python library functions for interacting with these systems. The architecture diagram shows MAADS-VIPER acting as both a source and sink to Confluent Platform/Cloud.