Incident Report: Spotify Outage on March 8, 2022

This post details a global outage on March 8, 2022, caused by an outage in Google Cloud Traffic Director and a bug in a gRPC client library. The incident led to users being unable to log back into Spotify apps. The immediate mitigation involved reverting affected systems to use DNS-based service discovery. Short-term actions include working with Google Cloud to understand the root cause and adding more monitoring and alerting for service discovery issues. The long-term investment is in further resiliency through monitoring, automatic error detection, and self-recovery.