Design sliding-window rate limiter with multi-keys
Design a Precise Sliding-Window Rate Limiter
Context
You are tasked with designing a rate limiter for an API that must enforce a true sliding-window limit. This means that at any given instant, only the traffic from the last T seconds counts towards the quota. The design process will involve two main parts: first, a global limiter, and then an extension to handle multi-dimensional limits.
Part A: Global Limit
Design and implement a precise sliding-window rate limiter that enforces a maximum of R requests within any rolling T-second window. This must be a true sliding window, not a fixed window or token bucket approach. For this part, you should specify:
- Public Interface: Define the inputs, return values, and error semantics for the rate limiter's API.
- Data Structures and State Storage: Describe the underlying data structures and how state will be stored, considering both single-host and distributed deployment options.
- Time Source and Resolution: Detail the chosen time source and the required resolution for accurate rate limiting.
- Time and Space Complexity: Analyze the time and space complexity per request for your proposed solution.
Part B: Per-Dimension Limits
Each incoming request includes two attributes: userId and userExperience. You must extend your design to concurrently enforce all of the following limits:
- Global Limit: R requests per T seconds (overall).
- Per-User Limit: U requests per T seconds, specific to each
userId. - Per-Experience Limit: X requests per T seconds, specific to each
userExperience.
For this extended design, explain your key choices regarding:
- Key/Counter Structure: How to organize keys and counters to support multiple dimensions without inadvertently double-counting requests.
- Efficient State Eviction: Strategies for efficiently evicting stale state (e.g., timestamps outside the T-second window).
- Distributed Deployment: How to deploy and scale the system in a distributed environment, addressing concerns such as sharding, coordination, clock skew, and idempotency.
- Testing and Edge Cases: Approaches to testing the correctness of the rate limiter, including handling bursts, boundary timestamps, and window rollovers.
AI interview prep
Core dilemma
Balancing the strict precision requirements of a true sliding window rate limiter with the performance, scalability, and consistency challenges of a distributed, multi-dimensional system.
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Design sliding-window rate limiter with multi-keys
Design a Precise Sliding-Window Rate Limiter
Context
You are tasked with designing a rate limiter for an API that must enforce a true sliding-window limit. This means that at any given instant, only the traffic from the last T seconds counts towards the quota. The design process will involve two main parts: first, a global limiter, and then an extension to handle multi-dimensional limits.
Part A: Global Limit
Design and implement a precise sliding-window rate limiter that enforces a maximum of R requests within any rolling T-second window. This must be a true sliding window, not a fixed window or token bucket approach. For this part, you should specify:
- Public Interface: Define the inputs, return values, and error semantics for the rate limiter's API.
- Data Structures and State Storage: Describe the underlying data structures and how state will be stored, considering both single-host and distributed deployment options.
- Time Source and Resolution: Detail the chosen time source and the required resolution for accurate rate limiting.
- Time and Space Complexity: Analyze the time and space complexity per request for your proposed solution.
Part B: Per-Dimension Limits
Each incoming request includes two attributes: userId and userExperience. You must extend your design to concurrently enforce all of the following limits:
- Global Limit: R requests per T seconds (overall).
- Per-User Limit: U requests per T seconds, specific to each
userId. - Per-Experience Limit: X requests per T seconds, specific to each
userExperience.
For this extended design, explain your key choices regarding:
- Key/Counter Structure: How to organize keys and counters to support multiple dimensions without inadvertently double-counting requests.
- Efficient State Eviction: Strategies for efficiently evicting stale state (e.g., timestamps outside the T-second window).
- Distributed Deployment: How to deploy and scale the system in a distributed environment, addressing concerns such as sharding, coordination, clock skew, and idempotency.
- Testing and Edge Cases: Approaches to testing the correctness of the rate limiter, including handling bursts, boundary timestamps, and window rollovers.