The Scary Thing About Automating Deploys

This post details the implementation of an automated deployment system, ReleaseBot, which replaces manual Deploy Commanders. The core of ReleaseBot's safety mechanism is anomaly detection, using z-scores and dynamic thresholds to monitor system metrics during deployments. This approach allows for faster and more vigilant detection of issues compared to human monitoring, thereby increasing deployment safety and freeing up valuable engineering time. The post explains the mathematical basis of z-scores and provides examples of their implementation in Python and PromQL.