Listening Together, Miles Apart

This post details the technical implementation of the 'Listening Together' campaign, which visualizes real-time global song plays. It describes the use of Google Cloud Pub/Sub for event ingestion, Apache Beam with Scio (Spotify's Scala API for Beam) and Google Dataflow for scalable stream processing, and windowing features to group plays within short timeframes. IP addresses are converted to locations using MaxMind databases. A backend service built on Spotify's Apollo framework, autoscaled in Kubernetes, pulls sessions from a new Pub/Sub topic to serve the website. The post highlights challenges in global synchronization and the efficiency of serving pre-calculated data.