How Uber Conquered Database Overload: The Journey from Static Rate-Limiting to Intelligent Load Management

This post details the development of an intelligent load manager for Uber's Docstore and Schemaless databases. It introduces a shift from quota-based rate limiting in the query engine to overload management within the stateful storage layer. Key contributions include the implementation of CoDel queues (Read, Write, Slow) with adaptive LIFO behavior for controlled delay, a Scorecard engine for per-tenant concurrency limits and fairness, and plug-in regulators (Write bytes, Partition key, Memory, Goroutines) to detect and mitigate specific overload conditions like I/O saturation, hot partition keys, and memory exhaustion. The post highlights improved availability and incident containment as key outcomes.