Changing the Wheels on a Moving Bus — Spotify’s Event Delivery Migration

This post details the migration of Spotify's Event Delivery Infrastructure (EDI) from a legacy, stateful, TSV-based system to a modern, cloud-native, Avro-based system. Key technical contributions include: addressing data loss issues by redesigning event delivery for mobile clients, improving control plane UX by schema-aware tooling and faster schema propagation, and migrating away from stateful services and legacy perimeter logging. A data transformation pipeline was implemented to handle legacy clients and enable a gradual transition to the new data model. The strategy involved finding early adopters, iterating on features, and solidifying interfaces to concurrently migrate internal users while building the new EDI.