Design a Telemetry Platform Ingesting Two Gigabytes per Second
Design a Telemetry Platform Ingesting Two Gigabytes per Second
Design an end-to-end telemetry logging platform for a robotics fleet that produces an aggregate 2 GB of raw event payloads per second at peak. The platform must support near-real-time reliability alerts, replayable historical analysis, and selective access for debugging.
Cover the following aspects:
- Collection on the robot
- Transport
- Ingestion
- Storage
- Processing
- Schemas
- Operations
Constraints & Assumptions:
- Treat GB as decimal gigabytes (10^9 bytes).
- Robots may lose connectivity and reconnect with buffered data.
- Telemetry contains a mix of critical low-rate events and high-rate sensor streams.
- Events can arrive late, out of order, or more than once.
- Raw evidence must remain replayable for a configurable retention period (duration not supplied).
- The platform must isolate tenants or fleets and protect sensitive operational data.
- Technologies can be chosen by capability, but the design should not depend on unverified product benchmarks.
AI interview prep
Core dilemma
Balancing high-throughput, potentially unreliable data ingestion with the need for near-real-time alerts, guaranteed data replayability, and efficient historical analysis, all while managing diverse data criticality and potential network disruptions.
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Design a Telemetry Platform Ingesting Two Gigabytes per Second
Design a Telemetry Platform Ingesting Two Gigabytes per Second
Design an end-to-end telemetry logging platform for a robotics fleet that produces an aggregate 2 GB of raw event payloads per second at peak. The platform must support near-real-time reliability alerts, replayable historical analysis, and selective access for debugging.
Cover the following aspects:
- Collection on the robot
- Transport
- Ingestion
- Storage
- Processing
- Schemas
- Operations
Constraints & Assumptions:
- Treat GB as decimal gigabytes (10^9 bytes).
- Robots may lose connectivity and reconnect with buffered data.
- Telemetry contains a mix of critical low-rate events and high-rate sensor streams.
- Events can arrive late, out of order, or more than once.
- Raw evidence must remain replayable for a configurable retention period (duration not supplied).
- The platform must isolate tenants or fleets and protect sensitive operational data.
- Technologies can be chosen by capability, but the design should not depend on unverified product benchmarks.