TechPrufer
Get started
Spotify
Spotify logoSpotify · Feature trail

Cassandra Orchestration Tool

3 posts · Nov 2014 – Aug 2018

What this trail covers

Spotify's Cassandra orchestration and management capabilities have evolved with the introduction and adoption of the Date-Tiered Compaction Strategy (DTCS). DTCS is optimized for time-series data, a common use case for Cassandra, by aligning SSTable age with data timestamps. This improves read performance by reducing the number of SSTables that need to be scanned for time-based queries. DTCS also aids in data removal by efficiently purging tombstones and TTL-expired data from older SSTables. While DTCS is highly effective for time-series and steady-rate write patterns, it is less suitable for highly out-of-order writes or scenarios with frequent repairs that introduce out-of-order timestamps. The post also emphasizes the importance of consistent timestamp formatting and client clock synchronization.

All Spotify trails
  1. Introducing cstar: The Spotify Cassandra orchestration tool, now open sourceThis post introduces cstar, a new open-source tool for orchestrating operations on large Cassandra clusters. It details the challenges of manual operations, the limitations of existing tools like Ansible and Fabric for Cassandra's topology, and presents cstar's features: topology-aware execution (ONE, TOPOLOGY, ALL), job control (pause/resume), and safe parallelization based on replication factors. The tool is built on paramiko for SSH and infers topology from a seed node.
  2. Cassandra: Data-Driven ConfigurationThis post details the use of the `cassandra-stress` tool for data-driven configuration and capacity planning of Spotify's Cassandra clusters. It walks through an example scenario of setting up a new service with a write-heavy, real-time pipeline and batch reads. The process involves defining a YAML profile for `cassandra-stress` that specifies the schema, data shape (column specifications), and queries. The post presents results from load tests on 3 and 6-node clusters, varying client connection counts, consistency levels (ONE vs. QUORUM), and column data sizes. The findings inform decisions about cluster size, column data limits, and the feasibility of increasing consistency levels while meeting latency SLAs.
  3. Date-Tiered Compaction in Apache CassandraThis post introduces and details the Date-Tiered Compaction Strategy (DTCS) for Apache Cassandra, a custom compaction strategy developed for time-series data. It explains the fundamental concepts of Cassandra's storage (SSTables, compaction) and contrasts DTCS with existing strategies (STCS, LCS). The core contribution is the design and benchmarking of DTCS, demonstrating its improved read performance for time-series data through better SSTable organization and its effectiveness in data purging. It also outlines the conditions under which DTCS is beneficial and when to avoid it, emphasizing the impact of out-of-order writes and timestamp consistency.