Rerouting the Stream: How Lyft Moved to the Apache Flink Operator

Lyft migrated its internal Flink Kubernetes operator to the open-source Apache Flink Kubernetes operator. This involved creating a translation layer to map legacy FlinkApplication CRDs to the Apache operator's FlinkDeployment CRDs, enabling incremental migration. The post details the challenges with the legacy operator, including maintenance burden, feature gaps (autoscaling, rollbacks), and dependency issues. It also discusses the migration's impact on savepoint fragility and the rigid resource model, and the additional work required for BlueGreen deployments, autoscaling, and autotuning.