Search Journey Towards Better Experimentation Practices
Feb 1, 2022Claire Detilleux, Sr. Data Scientist
Summary
This post details the Search team's journey towards better experimentation practices, complementing the existing experimentation platform. It outlines a three-step roadmap: 1. Individual experiment quality (templates for test specifications, experiment setups), 2. Cross-experiments quality (coordination strategies, guidelines to avoid user exhaustion), and 3. Measure total business impact (quarterly holdbacks for unbiased estimation). It also highlights the importance of 'constant injection of energy' through advocacy and knowledge transfer to embed these practices within teams.
More from the Experimentation Platform trail
- Why Spotify Is Not Using Bayesian A/B Testing | Spotify Engineering
- When Can LLMs Replace Humans in A/B Tests? | Spotify Engineering
- Better Experiments with LLM Evals — A funnel, not a fork | Spotify Engineering
- Why We Use Separate Tech Stacks for Personalization and Experimentation
- Beyond Winning: Spotify’s Experiments with Learning Framework