Design Scalable Notification Rate Limiter
Design Scalable Notification Rate Limiter
Design a production-grade system to rate limit notification sending requests. The system receives requests identified by a user_id and must make an atomic decision on whether to accept or reject the notification based on a hierarchical rate limiting policy.
Each user belongs to one team, and each team belongs to one company. The service must enforce exact rolling 10-minute limits at three distinct levels:
- User Level: A maximum of 3 accepted notifications per user within any rolling 10-minute window.
- Team Level: A maximum of 10 accepted notifications per team within any rolling 10-minute window.
- Company Level: A maximum of 20 accepted notifications per company within any rolling 10-minute window.
Your design should cover the following aspects:
- API and Request Flow: Define the external API for the rate limiter and describe the end-to-end request processing flow.
- State Management: Detail how the rate-limit state (e.g., counts, timestamps) will be stored and efficiently queried.
- Atomic Decision Making: Explain how to perform a single, atomic decision across all three hierarchical scopes (user, team, company) to ensure correctness.
- Scalability: Describe how the system will scale horizontally to handle a large number of requests across many stateless application servers.
- Hierarchy Changes & Cache Misses: Address strategies for handling cache misses when looking up user/team/company hierarchy data, and how changes in this hierarchy are propagated and managed.
- Advanced Considerations: Discuss how your design addresses challenges such as hot keys (e.g., a single user/team/company generating excessive requests), system failures, clock skew in a distributed environment, ensuring idempotency of operations, and providing comprehensive observability.
AI interview prep
Core dilemma
Balancing strict, atomic, multi-level rolling window rate limiting with high throughput, low latency, and fault tolerance in a distributed system, while managing hierarchical data consistency.
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Design Scalable Notification Rate Limiter
Design Scalable Notification Rate Limiter
Design a production-grade system to rate limit notification sending requests. The system receives requests identified by a user_id and must make an atomic decision on whether to accept or reject the notification based on a hierarchical rate limiting policy.
Each user belongs to one team, and each team belongs to one company. The service must enforce exact rolling 10-minute limits at three distinct levels:
- User Level: A maximum of 3 accepted notifications per user within any rolling 10-minute window.
- Team Level: A maximum of 10 accepted notifications per team within any rolling 10-minute window.
- Company Level: A maximum of 20 accepted notifications per company within any rolling 10-minute window.
Your design should cover the following aspects:
- API and Request Flow: Define the external API for the rate limiter and describe the end-to-end request processing flow.
- State Management: Detail how the rate-limit state (e.g., counts, timestamps) will be stored and efficiently queried.
- Atomic Decision Making: Explain how to perform a single, atomic decision across all three hierarchical scopes (user, team, company) to ensure correctness.
- Scalability: Describe how the system will scale horizontally to handle a large number of requests across many stateless application servers.
- Hierarchy Changes & Cache Misses: Address strategies for handling cache misses when looking up user/team/company hierarchy data, and how changes in this hierarchy are propagated and managed.
- Advanced Considerations: Discuss how your design addresses challenges such as hot keys (e.g., a single user/team/company generating excessive requests), system failures, clock skew in a distributed environment, ensuring idempotency of operations, and providing comprehensive observability.