Grab

Machine-learning predictive autoscaling for Flink

Oct 30, 2025Minh Nhat Nguyen
Machine-learning predictive autoscaling for Flink
SummaryAI, from the source post

Introduced a predictive autoscaler for Flink applications that uses machine learning to forecast workload and proactively adjust CPU resources. The system consists of a workload forecast model (time-series forecasting on Kafka throughput) and a resource prediction model (regression mapping Kafka throughput to Flink CPU usage). It aims to prevent reactive scaling spirals, restart spikes, and address limited parallelism constraints by enabling vertical scaling and deterministic resource allocation.

More from the Shadow Testing for Flink Deployments trail