Scalable User Privacy

This post details the development and implementation of Padlock, a key management service designed to enforce Spotify's privacy standard of encrypting all personal user data before persistence. Padlock manages user-specific keychains, using derived keys for services to encrypt/decrypt data. It is built for high scale (1M lookups/sec), low latency (p99 < 15ms), and high availability (99.95% SLO), utilizing Cassandra with Memcached caching and specific optimizations like LOCAL_ONE consistency reads and disabled read repair. The system allows for fine-grained data lifecycle management by categorizing data and assigning separate keys per category.