Lyft migrated its entire streaming fleet from an internally developed Flink Kubernetes operator to the open-source Apache Flink Kubernetes operator. This involved developing a translation layer to map legacy FlinkApplication custom resources to the Apache operator's FlinkDeployment custom resources, enabling incremental migration without disrupting existing deployment patterns. The migration addressed technical debt, closed feature gaps (like autoscaling and automatic rollbacks), and resolved issues with savepoint fragility and rigid resource models. Further work involved implementing FlinkBlueGreenDeployments for zero-downtime upgrades and integrating autoscaling and autotuning capabilities.
All Lyft trails