Experimenting with Machine Learning to Target In-App Messaging

This post details the application of uplift modeling and heterogeneous treatment effect prediction to target in-app messaging eligibility. It introduces a multi-headed model inspired by S-learners, T-learners, and Dragonnet, which predicts conditional average treatment effects (CATE) for two different outcomes. The model is trained simultaneously on shared and treatment-specific prediction heads. Offline policy evaluation is used to assess the model's impact on business metrics before A/B testing. The successful rollout resulted in significant improvements in user retention.