The Evolution of Cassandra Data Movement at Netflix

This post details the evolution of Netflix's Cassandra data movement from the monolithic Casspactor to a new layered architecture. The new system replaces Casspactor's fragile metadata dependencies with direct S3 reads and introduces a 'Connector Factory' model. This allows for purpose-built connectors for various Cassandra data abstractions (Key Value, Time Series, etc.) that operate on Spark DataFrames, eliminating intermediate table bloat, handling skewed partitions, and enabling time travel. The architecture also focuses on performance gains and significant cost savings.