Sign in to track progress, unlock prep layers, and browse your pipeline.Sign in
Design out-of-order windowed stream processor
Problem Statement
Design an event processor capable of handling an unbounded stream of events. Each event includes an id, timestamp, payload (string), and a checksum.
Constraints & Requirements
- Out-of-Order Arrival: Events may arrive out of their chronological timestamp order.
- Late Event Handling: Any event whose timestamp falls more than 60 seconds behind the latest processed event's timestamp (or a defined watermark) should be dropped.
- Event Validation: Each event must be validated using its
checksumupon arrival. Invalid events should be discarded. - Metric Calculation: The processor must continuously maintain and report the average length of the
payloadstrings for all valid events within the most recent 60-second window. - Continuous Updates: The system must support real-time updates to this average as new events arrive.
- Time Complexity: The total time complexity for processing
nevents must be no worse than O(n log n).
Deliverables
- Describe the core data structures used.
- Detail the strategy for handling late events.
- Explain how duplicate events (if any) are managed.
AI interview prep
Core dilemma
Efficiently maintaining a sliding window aggregate (average) over an unbounded, out-of-order event stream with strict time complexity and memory constraints, while handling late and duplicate events.
Mid-level signals3
Senior signals3
Red flags3
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Design out-of-order windowed stream processor
Problem Statement
Design an event processor capable of handling an unbounded stream of events. Each event includes an id, timestamp, payload (string), and a checksum.
Constraints & Requirements
- Out-of-Order Arrival: Events may arrive out of their chronological timestamp order.
- Late Event Handling: Any event whose timestamp falls more than 60 seconds behind the latest processed event's timestamp (or a defined watermark) should be dropped.
- Event Validation: Each event must be validated using its
checksumupon arrival. Invalid events should be discarded. - Metric Calculation: The processor must continuously maintain and report the average length of the
payloadstrings for all valid events within the most recent 60-second window. - Continuous Updates: The system must support real-time updates to this average as new events arrive.
- Time Complexity: The total time complexity for processing
nevents must be no worse than O(n log n).
Deliverables
- Describe the core data structures used.
- Detail the strategy for handling late events.
- Explain how duplicate events (if any) are managed.