Sign in to track progress, unlock prep layers, and browse your pipeline.Sign in
Design device logging system with offline upload
Distributed Message QueuesIdempotency and DeduplicationClient-side Persistence and Offline SyncSearch and Analytics DatabasesRate Limiting, Throttling, and Backpressur
Problem Description
Design a robust logging system capable of collecting logs from numerous client devices, reliably uploading them to a backend, and making them viewable in a web user interface.
Core Requirements
- Log Generation: Client devices continuously generate logs, which can be either structured or unstructured text.
- Offline Resilience: Devices may experience extended periods without internet connectivity. Upon reconnection, the system must reliably upload all accumulated logs.
- User Interface: A web-based UI, supported by appropriate APIs, is required to browse, search, and filter logs. Filtering capabilities should include criteria such as device ID, time range, and severity level.
- Freshness Target:
- For online devices, logs should appear in the UI within 3–5 minutes of being generated.
- For offline devices, logs should appear in the UI within approximately 3–5 minutes after successful reconnection and upload.
Follow-up Considerations
- Offline Handling: Detail how the design addresses scenarios where devices lack internet connectivity for prolonged periods, specifically covering aspects like offline buffering, data reliability, deduplication, and maintaining log order.
- Traffic Surges: Explain how the system handles simultaneous log uploads from a large number of devices (e.g., a daily 6 PM surge), focusing on strategies such as throttling, autoscaling, load shedding, and prioritization mechanisms.
AI interview prep
Core dilemma
Balancing reliable, potentially high-volume, bursty ingestion from numerous unreliable clients with near real-time searchability and analytics, all while managing client-side complexity and backend surge capacity.
Mid-level signals3
Senior signals3
Red flags3
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Design device logging system with offline upload
Distributed Message QueuesIdempotency and DeduplicationClient-side Persistence and Offline SyncSearch and Analytics DatabasesRate Limiting, Throttling, and Backpressur
Problem Description
Design a robust logging system capable of collecting logs from numerous client devices, reliably uploading them to a backend, and making them viewable in a web user interface.
Core Requirements
- Log Generation: Client devices continuously generate logs, which can be either structured or unstructured text.
- Offline Resilience: Devices may experience extended periods without internet connectivity. Upon reconnection, the system must reliably upload all accumulated logs.
- User Interface: A web-based UI, supported by appropriate APIs, is required to browse, search, and filter logs. Filtering capabilities should include criteria such as device ID, time range, and severity level.
- Freshness Target:
- For online devices, logs should appear in the UI within 3–5 minutes of being generated.
- For offline devices, logs should appear in the UI within approximately 3–5 minutes after successful reconnection and upload.
Follow-up Considerations
- Offline Handling: Detail how the design addresses scenarios where devices lack internet connectivity for prolonged periods, specifically covering aspects like offline buffering, data reliability, deduplication, and maintaining log order.
- Traffic Surges: Explain how the system handles simultaneous log uploads from a large number of devices (e.g., a daily 6 PM surge), focusing on strategies such as throttling, autoscaling, load shedding, and prioritization mechanisms.