Design an analytics dashboard for an AI chat app
Design an Analytics Dashboard for an AI Chat App
Design an analytics dashboard for a ChatGPT-like AI chat product. This dashboard will be used by Product, Engineering, and Support teams to monitor product health and usage.
Core Use Cases
- Near real-time monitoring of:
- Usage: DAU/WAU/MAU, sessions, messages per user, retention cohorts.
- Performance: end-to-end latency (P50/P95/P99), streaming token latency, time-to-first-token.
- Reliability: error rate, timeouts, rate limits.
- Quality proxies: user feedback (thumbs up/down), conversation abandonment.
- Cost: tokens generated, $ cost by model/tenant.
- Slice metrics by dimensions: model, region, platform, tenant, app version.
- Support time ranges (last 15m, 1h, 24h, custom) and drill-down capabilities.
Non-Functional Requirements
- Data freshness targets:
- Operational metrics: ~1–5 minutes.
- Business/retention metrics: hourly/daily acceptable.
- High availability; the dashboard should degrade gracefully.
- Privacy/security: PII handling, access control, audit logs.
Deliverable
Propose an end-to-end system design, covering instrumentation, data ingestion, storage, aggregation, query patterns, and the dashboard serving layer.
AI interview prep
Core dilemma
Balancing the need for near real-time operational metrics with the complexity and cost of processing and storing vast amounts of event data for business and retention analytics.
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Design an analytics dashboard for an AI chat app
Design an Analytics Dashboard for an AI Chat App
Design an analytics dashboard for a ChatGPT-like AI chat product. This dashboard will be used by Product, Engineering, and Support teams to monitor product health and usage.
Core Use Cases
- Near real-time monitoring of:
- Usage: DAU/WAU/MAU, sessions, messages per user, retention cohorts.
- Performance: end-to-end latency (P50/P95/P99), streaming token latency, time-to-first-token.
- Reliability: error rate, timeouts, rate limits.
- Quality proxies: user feedback (thumbs up/down), conversation abandonment.
- Cost: tokens generated, $ cost by model/tenant.
- Slice metrics by dimensions: model, region, platform, tenant, app version.
- Support time ranges (last 15m, 1h, 24h, custom) and drill-down capabilities.
Non-Functional Requirements
- Data freshness targets:
- Operational metrics: ~1–5 minutes.
- Business/retention metrics: hourly/daily acceptable.
- High availability; the dashboard should degrade gracefully.
- Privacy/security: PII handling, access control, audit logs.
Deliverable
Propose an end-to-end system design, covering instrumentation, data ingestion, storage, aggregation, query patterns, and the dashboard serving layer.