How Uber Protects Against Retry Storms

This post introduces a novel approach to mitigating retry storms by establishing 'error ownership'. It details a mechanism that correlates inbound service errors with outbound call failures, using a decision logic to determine if a service is the true 'owner' of an error. This allows for conditional retries, preventing upstream services from retrying when the error originates deeper in the dependency chain. The system leverages a Service Dependency Analysis Solution and introduces an 'x-uber-error-claim' header to propagate error ownership information. The post also discusses the challenges of coincidental errors and how the system can be adapted to avoid suppressing legitimate retries.