Failing Forward — How We Grow from Incidents
This post analyzes incidents affecting Spotify for Artists in 2021, quantifying their impact on productivity and preventability. It highlights that most incidents have a moderate to high productivity impact, with 23% affecting more than one responder for a significant portion of a day. The analysis categorizes incidents by preventability, with most scoring 3 or higher, indicating localized failures that could be prevented or mitigated. A key finding is the significant reduction in Time to Recover (TTR) for incidents where synthetic tests were in place, showing a 10x faster recovery time. The post also discusses the challenges in accurately recording incident timelines and the need for better systems and incentives to capture this data, referencing DORA metrics.