Design Experiment to Measure Airport Surge-Pricing Impact
Experiment Design: Measuring Causal Impact of Airport Surge-Pricing Push Notifications
Context
You are operating a two-sided ride-hailing marketplace. A new system has been implemented to send push notifications to eligible drivers when an airport experiences surge pricing. The objective of these notifications is to increase driver supply at the airport. A critical challenge is that drivers within and around the airport can see and respond to these pushes at overlapping times, making interference (spillovers) between treated and untreated drivers a plausible and significant concern.
Task
Design a robust experiment to causally measure the impact of these push notifications on driver supply at the airport, specifically addressing and mitigating potential interference effects.
Your design should explicitly cover:
- Experimental Design: Define the randomization unit(s), necessary holdouts, appropriate timing windows, and any clustering strategies.
- Success Metrics: Identify the primary and secondary metrics that will be tracked to evaluate the experiment's success.
- Spillover Handling: Detail how you will detect and account for interference effects on untreated drivers.
- Causal Impact Identification: Explain the methodologies you will employ to identify the causal impact in the presence of interference (e.g., geographic clustering, holdout zones, difference-in-differences, network interference adjustments).
AI interview prep
Core dilemma
The central engineering conflict is how to isolate the causal effect of a treatment (push notifications) in a highly interconnected system where the actions of treated units (drivers receiving pushes) directly influence untreated units (nearby drivers not receiving pushes), thereby violating the Stable Unit Treatment Value Assumption (SUTVA). This requires sophisticated experimental design and analytical techniques beyond simple A/B testing.
3 tradeoffs and 2 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Design Experiment to Measure Airport Surge-Pricing Impact
Experiment Design: Measuring Causal Impact of Airport Surge-Pricing Push Notifications
Context
You are operating a two-sided ride-hailing marketplace. A new system has been implemented to send push notifications to eligible drivers when an airport experiences surge pricing. The objective of these notifications is to increase driver supply at the airport. A critical challenge is that drivers within and around the airport can see and respond to these pushes at overlapping times, making interference (spillovers) between treated and untreated drivers a plausible and significant concern.
Task
Design a robust experiment to causally measure the impact of these push notifications on driver supply at the airport, specifically addressing and mitigating potential interference effects.
Your design should explicitly cover:
- Experimental Design: Define the randomization unit(s), necessary holdouts, appropriate timing windows, and any clustering strategies.
- Success Metrics: Identify the primary and secondary metrics that will be tracked to evaluate the experiment's success.
- Spillover Handling: Detail how you will detect and account for interference effects on untreated drivers.
- Causal Impact Identification: Explain the methodologies you will employ to identify the causal impact in the presence of interference (e.g., geographic clustering, holdout zones, difference-in-differences, network interference adjustments).