Session Management and Realtime Infrastructure Resilience

Discord's session management and realtime infrastructure have been enhanced to improve resilience against cascading failures. Following a significant outage caused by a configuration update that led to a large number of session management servers shutting down, the team has analyzed system behavior under sudden load. Improvements focus on preventing similar widespread degradation by strengthening downstream systems and identifying and mitigating new bottlenecks that emerge during load spikes. This post details how a 17% loss of session management servers overwhelmed a routing service, leading to voice and video degradation, and the subsequent analysis and improvements made to prevent recurrence.