Transform and aggregate messy event data
Problem: Transform and Aggregate Messy Event Data
This take-home project requires you to process user and event data using the pandas library, specifically leveraging vectorized operations without explicit loops. The objective is to clean, combine, and aggregate this data to produce country and plan-level metrics for a specific date: 2025-08-31.
Two initial DataFrames are provided:
users: Containsuser_id,signup_date,plan,monthly_fee, andcountry.events_august: Containsuser_id,event_time,event_type, andamount.
A third DataFrame, events_august_extra, has the same fields as events_august but with a shuffled column order.
Tasks:
- Data Type Conversion and Cleaning:
- Convert
users.signup_dateand allevent_timecolumns to pandas datetime objects. - For
monthly_feeandamountcolumns, strip any currency symbols and whitespace, then coerce them to float type. Any conversion errors should result inNaNvalues.
- Convert
- Event Data Consolidation:
- Align the columns of
events_augustandevents_august_extra. - Concatenate these two DataFrames into a single
events_allDataFrame.
- Align the columns of
- Data Integration:
- Join the
usersDataFrame with theevents_allDataFrame usinguser_idas the key.
- Join the
- Metric Computation:
- For the date 2025-08-31 only, compute the following metrics, grouped by
countryandplan:active_users: The count of distinct users who had at least one 'login' event.purchasers: The count of distinct users who had at least one 'purchase' event.purchases_count: The total number of 'purchase' events.revenue: The sum ofamountfor all 'purchase' events.
- For the date 2025-08-31 only, compute the following metrics, grouped by
- Output Format:
- Return a tidy DataFrame with the columns
[date, country, plan, active_users, purchasers, purchases_count, revenue]. - The resulting DataFrame should be sorted first by
countryin ascending order, then byplanin ascending order. - The solution must explicitly state any assumptions made regarding missing
amountvalues and timezone handling.
- Return a tidy DataFrame with the columns
AI interview prep
Core dilemma
The central engineering conflict is balancing efficient, vectorized data manipulation for complex aggregations with robust handling of diverse, messy, and potentially malformed input data, all while adhering to strict performance and memory constraints (implicit in 'no loops' for large datasets).
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Transform and aggregate messy event data
Problem: Transform and Aggregate Messy Event Data
This take-home project requires you to process user and event data using the pandas library, specifically leveraging vectorized operations without explicit loops. The objective is to clean, combine, and aggregate this data to produce country and plan-level metrics for a specific date: 2025-08-31.
Two initial DataFrames are provided:
users: Containsuser_id,signup_date,plan,monthly_fee, andcountry.events_august: Containsuser_id,event_time,event_type, andamount.
A third DataFrame, events_august_extra, has the same fields as events_august but with a shuffled column order.
Tasks:
- Data Type Conversion and Cleaning:
- Convert
users.signup_dateand allevent_timecolumns to pandas datetime objects. - For
monthly_feeandamountcolumns, strip any currency symbols and whitespace, then coerce them to float type. Any conversion errors should result inNaNvalues.
- Convert
- Event Data Consolidation:
- Align the columns of
events_augustandevents_august_extra. - Concatenate these two DataFrames into a single
events_allDataFrame.
- Align the columns of
- Data Integration:
- Join the
usersDataFrame with theevents_allDataFrame usinguser_idas the key.
- Join the
- Metric Computation:
- For the date 2025-08-31 only, compute the following metrics, grouped by
countryandplan:active_users: The count of distinct users who had at least one 'login' event.purchasers: The count of distinct users who had at least one 'purchase' event.purchases_count: The total number of 'purchase' events.revenue: The sum ofamountfor all 'purchase' events.
- For the date 2025-08-31 only, compute the following metrics, grouped by
- Output Format:
- Return a tidy DataFrame with the columns
[date, country, plan, active_users, purchasers, purchases_count, revenue]. - The resulting DataFrame should be sorted first by
countryin ascending order, then byplanin ascending order. - The solution must explicitly state any assumptions made regarding missing
amountvalues and timezone handling.
- Return a tidy DataFrame with the columns