Experimenting at Scale, the Spotify Home Way
Jun 1, 2023Nik Goyle, Product Manager
Summary
This post details the specific tooling and coordination mechanisms used by Spotify's Home surface to run experiments at scale. It introduces Home Config for simplified configuration management by less technical users, highlights the integration of the Experimentation Platform (EP) within Backstage for launching and monitoring, and describes Home QA for debugging and testing. For coordination, it explains the Experiment Tracker for prioritizing experiments and EVA (Experiment Validation Assistant) for automated validation of A/B tests to ensure result validity and reduce misconfigurations. It also mentions the upcoming introduction of Interleaving.
More from the Experimentation Platform trail
- Why Spotify Is Not Using Bayesian A/B Testing | Spotify Engineering
- When Can LLMs Replace Humans in A/B Tests? | Spotify Engineering
- Better Experiments with LLM Evals — A funnel, not a fork | Spotify Engineering
- Why We Use Separate Tech Stacks for Personalization and Experimentation
- Beyond Winning: Spotify’s Experiments with Learning Framework