Count Blocked Rate-Limited Requests
Count Blocked Rate-Limited Requests
You are given an in-memory log of web requests, already sorted by timestamp in ascending order. Each log entry contains:
timestamp: An ISO-8601 UTC timestamp.request_ip: The client's IPv4 address.host: The hostname being requested.
You may assume the entire log fits into memory.
A request is evaluated against a sliding window of T seconds. For a request occurring at time t, only earlier requests with timestamps strictly within the interval (t - T, t) are considered. Requests that occurred exactly T seconds earlier are not included in the window.
Important Rules:
- A request that is blocked must still be counted in the history for future rate-limiting evaluations.
- If multiple requests share the same timestamp, they should be processed in their given input order.
- If a request violates multiple rate-limiting rules (in Part 2), it should be counted as a single blocked request, not multiple.
Part 1: Per-IP Rate Limiting
Implement logic to count the total number of requests that would be blocked by the following rule:
- A request from IP
xis blocked if there have already been at leastIPReqMaxrequests from the same IP within the previousTseconds.
Formally, the current request is blocked when:
count(previous requests from the same IP in (t - T, t)) >= IPReqMax
Example for Part 1
Input log:
timestamp,request_ip,host
2024-01-01T00:00:00+00:00,127.0.0.1,example1.com
2024-01-01T00:00:01+00:00,127.0.0.1,example2.com
2024-01-01T00:01:02+00:00,127.0.0.1,example1.com
2024-01-01T00:01:03+00:00,127.0.0.1,example2.com
2024-01-01T00:01:04+00:00,127.0.0.1,example1.com
2024-01-01T00:02:04+00:00,127.0.0.1,example1.com
With IPReqMax = 1 and T = 60:
- Request 1 (00:00:00) -> succeed
- Request 2 (00:00:01) -> blocked
- Request 3 (00:01:02) -> succeed
- Request 4 (00:01:03) -> blocked
- Request 5 (00:01:04) -> blocked
- Request 6 (00:02:04) -> succeed
Total blocked requests: 3.
Part 2: Per-IP and Per-(IP, Host) Rate Limiting
Now, an additional rule is introduced:
- A request from IP
xto hosthis also blocked if there have already been at leastIPHostReqMaxrequests from the same IP to the same host within the previousTseconds.
A request is considered blocked if either of these conditions is true:
count(previous requests from the same IP in (t - T, t)) >= IPReqMaxcount(previous requests from the same IP and same host in (t - T, t)) >= IPHostReqMax
As before, blocked requests still contribute to both counters for future evaluations.
Example for Part 2
Input log:
timestamp,request_ip,host
2024-01-01T00:00:00+00:00,127.0.0.1,example1.com
2024-01-01T00:00:01+00:00,127.0.0.1,example2.com
2024-01-01T00:00:10+00:00,127.0.0.1,example1.com
2024-01-01T00:00:11+00:00,127.0.0.1,example3.com
2024-01-01T00:01:04+00:00,127.0.0.1,example3.com
2024-01-01T00:01:11+00:00,127.0.0.1,example1.com
With IPReqMax = 3, IPHostReqMax = 1, and T = 60:
- Request 1 (00:00:00, 127.0.0.1, example1.com) -> succeed
- Request 2 (00:00:01, 127.0.0.1, example2.com) -> succeed
- Request 3 (00:00:10, 127.0.0.1, example1.com) -> blocked by per-(IP, host) rule
- Request 4 (00:00:11, 127.0.0.1, example3.com) -> blocked by per-IP rule
- Request 5 (00:01:04, 127.0.0.1, example3.com) -> blocked by per-(IP, host) rule
- Request 6 (00:01:11, 127.0.0.1, example1.com) -> succeed
Total blocked requests: 3.
Task
Your goal is to write an efficient function or program that, given the sorted request log and the parameters T, IPReqMax, and optionally IPHostReqMax, returns:
- The total number of blocked requests under the per-IP rule only (Part 1).
- The total number of blocked requests under the combined per-IP and per-(IP, host) rules (Part 2).
The solution should be efficient for large logs that fit into memory.
AI interview prep
Core dilemma
Efficiently maintaining and querying multiple, concurrent sliding windows of request counts across different aggregation keys (IP, IP+Host) over a large, time-sorted dataset, while ensuring correct eviction logic and handling edge cases like equal timestamps and blocked requests contributing to future counts.
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Count Blocked Rate-Limited Requests
Count Blocked Rate-Limited Requests
You are given an in-memory log of web requests, already sorted by timestamp in ascending order. Each log entry contains:
timestamp: An ISO-8601 UTC timestamp.request_ip: The client's IPv4 address.host: The hostname being requested.
You may assume the entire log fits into memory.
A request is evaluated against a sliding window of T seconds. For a request occurring at time t, only earlier requests with timestamps strictly within the interval (t - T, t) are considered. Requests that occurred exactly T seconds earlier are not included in the window.
Important Rules:
- A request that is blocked must still be counted in the history for future rate-limiting evaluations.
- If multiple requests share the same timestamp, they should be processed in their given input order.
- If a request violates multiple rate-limiting rules (in Part 2), it should be counted as a single blocked request, not multiple.
Part 1: Per-IP Rate Limiting
Implement logic to count the total number of requests that would be blocked by the following rule:
- A request from IP
xis blocked if there have already been at leastIPReqMaxrequests from the same IP within the previousTseconds.
Formally, the current request is blocked when:
count(previous requests from the same IP in (t - T, t)) >= IPReqMax
Example for Part 1
Input log:
timestamp,request_ip,host
2024-01-01T00:00:00+00:00,127.0.0.1,example1.com
2024-01-01T00:00:01+00:00,127.0.0.1,example2.com
2024-01-01T00:01:02+00:00,127.0.0.1,example1.com
2024-01-01T00:01:03+00:00,127.0.0.1,example2.com
2024-01-01T00:01:04+00:00,127.0.0.1,example1.com
2024-01-01T00:02:04+00:00,127.0.0.1,example1.com
With IPReqMax = 1 and T = 60:
- Request 1 (00:00:00) -> succeed
- Request 2 (00:00:01) -> blocked
- Request 3 (00:01:02) -> succeed
- Request 4 (00:01:03) -> blocked
- Request 5 (00:01:04) -> blocked
- Request 6 (00:02:04) -> succeed
Total blocked requests: 3.
Part 2: Per-IP and Per-(IP, Host) Rate Limiting
Now, an additional rule is introduced:
- A request from IP
xto hosthis also blocked if there have already been at leastIPHostReqMaxrequests from the same IP to the same host within the previousTseconds.
A request is considered blocked if either of these conditions is true:
count(previous requests from the same IP in (t - T, t)) >= IPReqMaxcount(previous requests from the same IP and same host in (t - T, t)) >= IPHostReqMax
As before, blocked requests still contribute to both counters for future evaluations.
Example for Part 2
Input log:
timestamp,request_ip,host
2024-01-01T00:00:00+00:00,127.0.0.1,example1.com
2024-01-01T00:00:01+00:00,127.0.0.1,example2.com
2024-01-01T00:00:10+00:00,127.0.0.1,example1.com
2024-01-01T00:00:11+00:00,127.0.0.1,example3.com
2024-01-01T00:01:04+00:00,127.0.0.1,example3.com
2024-01-01T00:01:11+00:00,127.0.0.1,example1.com
With IPReqMax = 3, IPHostReqMax = 1, and T = 60:
- Request 1 (00:00:00, 127.0.0.1, example1.com) -> succeed
- Request 2 (00:00:01, 127.0.0.1, example2.com) -> succeed
- Request 3 (00:00:10, 127.0.0.1, example1.com) -> blocked by per-(IP, host) rule
- Request 4 (00:00:11, 127.0.0.1, example3.com) -> blocked by per-IP rule
- Request 5 (00:01:04, 127.0.0.1, example3.com) -> blocked by per-(IP, host) rule
- Request 6 (00:01:11, 127.0.0.1, example1.com) -> succeed
Total blocked requests: 3.
Task
Your goal is to write an efficient function or program that, given the sorted request log and the parameters T, IPReqMax, and optionally IPHostReqMax, returns:
- The total number of blocked requests under the per-IP rule only (Part 1).
- The total number of blocked requests under the combined per-IP and per-(IP, host) rules (Part 2).
The solution should be efficient for large logs that fit into memory.