Sign in to track progress, unlock prep layers, and browse your pipeline.Sign in
Diagnose a 10% DAU drop
On-Call Incident: Yahoo Mail DAU Down 10%
An incident has been declared for Yahoo Mail: the Daily Active User (DAU) count has dropped by 10% on 2025-09-01 compared to the 7-day baseline (2025-08-25 to 2025-08-31).
Assume the following:
- All times are UTC.
- Yahoo Mail is a global product.
- DAU is the canonical daily active user metric.
- 2025-09-01 is a Monday and a US holiday (Labor Day), which may or may not be a contributing factor.
Task
Outline an end-to-end plan to address this incident, covering the following stages:
1. Verify the Drop
- Clarify the precise definition and identity of the DAU metric.
- Ensure proper bot and employee filters are applied, and confirm experiment/treatment exclusions.
- Check the health of log ingestion and ETL pipelines, and verify data freshness.
- Segment the DAU drop by platform (web, iOS, Android) and app version buckets.
2. Isolate Internal vs. External Causes
- Utilize a metric tree decomposition: DAU = New Users + Returning Users; Returning Users = Retained Users + Resurrected Users.
- Segment the DAU by various dimensions: geographical region, device type, app version, notification eligibility, spam-folder rate, inbox-load latency, login failures, SMTP delivery latency, recent outages, release rollouts, and paywall/ads interactions.
3. Run Quick SQL/Python Checks
- Identify the top three contributing segments by absolute DAU loss when compared to the 7-day baseline.
- Estimate each identified segment's proportional contribution to the total 10% DAU loss.
4. Propose a Causal Validation
- Design a causal test or quasi-experiment to validate the suspected cause. Examples include:
- A/B rollback.
- Holdout by app version.
- Difference-in-differences analysis with synthetic controls using unaffected geographical regions.
5. Recommend Mitigations and Monitoring
- Propose immediate mitigations to address the identified issue.
- Develop a comprehensive monitoring plan, including alert thresholds, guardrail metrics (e.g., open rate, send success, latency), and clear rollback criteria.
- Be specific about the data required, the success metrics for mitigation, and the precise order of operations within the first two hours of the incident response.
AI interview prep
Core dilemma
Balancing the urgency of diagnosing and mitigating a significant DAU drop with the rigor required for accurate causal inference in a complex, global system, often with imperfect or delayed data.
Mid-level signals3
Senior signals4
Red flags4
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Diagnose a 10% DAU drop
On-Call Incident: Yahoo Mail DAU Down 10%
An incident has been declared for Yahoo Mail: the Daily Active User (DAU) count has dropped by 10% on 2025-09-01 compared to the 7-day baseline (2025-08-25 to 2025-08-31).
Assume the following:
- All times are UTC.
- Yahoo Mail is a global product.
- DAU is the canonical daily active user metric.
- 2025-09-01 is a Monday and a US holiday (Labor Day), which may or may not be a contributing factor.
Task
Outline an end-to-end plan to address this incident, covering the following stages:
1. Verify the Drop
- Clarify the precise definition and identity of the DAU metric.
- Ensure proper bot and employee filters are applied, and confirm experiment/treatment exclusions.
- Check the health of log ingestion and ETL pipelines, and verify data freshness.
- Segment the DAU drop by platform (web, iOS, Android) and app version buckets.
2. Isolate Internal vs. External Causes
- Utilize a metric tree decomposition: DAU = New Users + Returning Users; Returning Users = Retained Users + Resurrected Users.
- Segment the DAU by various dimensions: geographical region, device type, app version, notification eligibility, spam-folder rate, inbox-load latency, login failures, SMTP delivery latency, recent outages, release rollouts, and paywall/ads interactions.
3. Run Quick SQL/Python Checks
- Identify the top three contributing segments by absolute DAU loss when compared to the 7-day baseline.
- Estimate each identified segment's proportional contribution to the total 10% DAU loss.
4. Propose a Causal Validation
- Design a causal test or quasi-experiment to validate the suspected cause. Examples include:
- A/B rollback.
- Holdout by app version.
- Difference-in-differences analysis with synthetic controls using unaffected geographical regions.
5. Recommend Mitigations and Monitoring
- Propose immediate mitigations to address the identified issue.
- Develop a comprehensive monitoring plan, including alert thresholds, guardrail metrics (e.g., open rate, send success, latency), and clear rollback criteria.
- Be specific about the data required, the success metrics for mitigation, and the precise order of operations within the first two hours of the incident response.