TechPrufer
Get started
Dropbox
Dropbox logoDropbox · Feature trail

Experimentation

1 post · Jan 2023

What this trail covers

Dropbox has developed and deployed a machine learning system to optimize payment renewal attempts, reducing involuntary churn and improving customer retention. This system moves beyond static, rule-based retry logic to dynamically predict the best times to charge customers during renewal failures, leading to increased payment success rates and a more efficient billing process. The ML approach also simplifies the overall billing system complexity. This post details the development of Expected Revenue (XR) as a metric for A/B experiments, using Gradient Boosted Trees in TensorFlow to predict user lifetime value and enable faster, more accurate experiment analysis. The system leverages Airflow, Hadoop on AWS, Hive, Spark, and S3 for its architecture.

All Dropbox trails
  1. Accelerating our A/B experiments with machine learningThis post introduces the concept of Expected Revenue (XR) as a machine learning-driven metric for analyzing A/B experiments at Dropbox. It details how XR is calculated using Gradient Boosted Trees in TensorFlow to predict a user's two-year lifetime value, accounting for conversion probability, retention, and plan switching. The system leverages Airflow for orchestration, Hadoop on AWS for the backend, Hive for data storage, and Spark for evaluation. A key contribution is the method for calculating XR lift as a ratio to reduce systematic uncertainty, making experimental conclusions more reliable.