The Rise (and Lessons Learned) of ML Models to Personalize Content on Home (Part II)

This post details the evolution of ML models for personalizing content on Spotify's Home feed. It covers the transition from a siloed experimentation platform to integrated Kubeflow clusters, highlighting challenges with custom evaluation metrics and comparing ML models against heuristic baselines. The post also discusses the development of custom dashboards for qualitative evaluation of recommendations and the struggles of automated model retraining and deployment, emphasizing the need for frequent retraining for certain models and the technical debt incurred by others. The authors contributed to the Python SDK for Kubeflow to support custom Python evaluations and developed a dashboard for manual evaluation of recommendations.