How we built rate limiting capable of scaling to millions of domains

This post details the implementation of Cloudflare's rate limiting system, designed to scale to millions of domains. It describes the challenges of distributed rate limiting across an anycast network, the use of a Twemproxy cluster with memcached for shared counters within each PoP, and the adoption of a sliding window algorithm for accurate rate approximation. The system leverages asynchronous counter increments and in-memory caching of mitigation states to handle high-volume L7 attacks efficiently without impacting legitimate traffic.