Balancing Safety and Velocity in CI/CD at Slack

This post details a significant shift in Slack's CI/CD workflow for the Webapp monorepo, moving from predominantly pre-merge testing to a multi-tiered post-merge testing strategy. Key technical changes include the introduction of a post-merge pipeline (executing ~10% of E2E tests) and a regression pipeline (executing remaining E2E tests in batches bi-hourly). This reduced test turnaround time by over 40% and test flakiness per PR by over 90%. The post also describes triage changes, including creating dedicated Slack channels for alerts and assigning test ownership to business areas, and the design of human-centric alerts using a finite state machine to detect consecutive failures.