TechPrufer
Get started
ScyllaDB
ScyllaDB logoScyllaDB · Feature trail

System Scalability Design

17 posts · May 2026 – Aug 2026

What this trail covers

ScyllaDB X Cloud offers elastic scaling for variable workloads, allowing clusters to scale from 10K to 1M ops/sec and back down in minutes with single-digit millisecond P99 latency. This is achieved through automatic tablet streaming and rebalancing, enabling rapid capacity adjustments to match actual usage without overprovisioning. The system supports mixed instance sizes and maintains performance during scaling events. The Terraform provider allows for infrastructure-as-code management of these scaling operations.

All ScyllaDB trails
  1. Building a New Rust Driver for ScyllaDB’s DynamoDB API – with 58% More ThroughputThis post details the development of a new Rust driver for ScyllaDB's DynamoDB API (Alternator). The driver leverages the AWS SDK's Interceptor framework to inject optimization logic tailored for ScyllaDB's architecture. Key optimizations include intelligent load balancing by understanding cluster topology and routing requests to appropriate nodes, achieving ~58% higher throughput and lower P99 latencies compared to the AWS SDK driver. Additionally, ScyllaDB's Rust benchmarking tool, Latte, was extended to support the DynamoDB API for performance measurement.
  2. How Sprig Replaced Postgres, ClickHouse & Redis…with 4-8x Better LatencyThis post details Sprig's migration from Postgres, ClickHouse, and Redis to ScyllaDB. It outlines the challenges faced with their initial relational database setup due to rapid growth (1.3T events, 75B attributes), the limitations encountered with Aurora Postgres (write throughput saturation, instance limits), and the subsequent adoption of ScyllaDB to achieve 4-8x better latency. The technical details focus on the data volume, event processing rate (20k-40k events/sec), and the need for low-latency decision-making for in-product surveys.
  3. How Medium Powers Real-Time Recommendations at 1M OPSMedium migrated its feature store from relational features to list features in ScyllaDB to handle 1M OPS for real-time recommendations. This involved redesigning the data model to use list features for storing time-series user activity, improving query performance and scalability for their recommendation system.
  4. DDIA 2nd Edition Excerpt: On ScalabilityThis post is an excerpt from 'Designing Data-Intensive Applications Second Edition' and discusses the fundamental concepts of scalability in the context of data-intensive systems. It defines scalability as a system's ability to handle increased load and emphasizes that it's a nuanced topic, not a simple yes/no attribute. The authors advise against premature optimization for hypothetical future scale, suggesting that engineers should focus on simplicity and flexibility in early stages. They propose that scalability becomes a critical concern when an application's success leads to significant load growth, necessitating an understanding of performance bottlenecks and the options for adding computing resources. The post also touches upon the importance of understanding current load metrics to inform future scaling decisions.
  5. Lessons Learned from Real-World NoSQL Database MigrationsThis post provides practical insights into NoSQL database migrations, specifically detailing a migration from DynamoDB to ScyllaDB. It elaborates on the challenges of out-of-order writes, the strategy of dual-writing during migration, and the importance of understanding subtle differences between similar databases. The author also mentions the use of ScyllaDB Migrator and the considerations for record versioning and data compression.
  7. What Matters Most for NoSQL MigrationsThis post elaborates on NoSQL database migration strategies, detailing the importance of schema tuning, data migration, and data validation. It outlines online and offline migration approaches and discusses techniques for capturing and replaying database changes using tools like Dynamo Streams, Kafka, and CDC. The post also highlights the significance of understanding database differences and data modeling for successful migrations, referencing a masterclass on NoSQL data modeling for performance.
  8. Build Durable Chat Memory for RAG Using ScyllaDB and LangChainThis post demonstrates integrating ScyllaDB Vector Search with LangChain for RAG use cases and using ScyllaDB as a durable conversation memory within LangChain. It highlights ScyllaDB's benefits for AI applications, including high availability, predictable P99 latency, and horizontal scalability, and mentions the reuse of the Cassandra connector for integration.
  9. Agent Memory at Monster Scale with Mem0 and ScyllaDB CloudThis post introduces the integration of Mem0's memory management framework with ScyllaDB Cloud for large-scale AI agent applications. It highlights ScyllaDB Cloud's suitability for global, high-throughput, low-latency vector search, emphasizing its high availability and P99 guarantees as critical for the persistence layer of AI agent memory. The post details how Mem0 extracts and stores facts as vector embeddings, and how ScyllaDB Cloud serves as the distributed database infrastructure for this memory layer.
  10. ScyllaDB vs Aerospike, Wide-Column vs. Key/ValueThis post compares ScyllaDB's wide-column (KKV) data model with Aerospike's key-value (KV) model. It explains the architectural differences, focusing on clustering keys in ScyllaDB enabling advanced access patterns like time-series data and messaging. The post argues that ScyllaDB's flexibility does not compromise performance and can even outperform Aerospike in specific workloads, despite Aerospike's optimized KV design. It also mentions ScyllaDB's development of a pure K/V option.
  11. How ScyllaDB’s Trie-Based Index Delivers Up to 3X More ThroughputIntroduced a trie-based SSTable index format to replace the legacy three-layer lookup (me/md) format. This change optimizes cache efficiency, reduces disk I/O, and lowers memory overhead by using a prefix tree structure for indexing. Benchmarks demonstrate up to 3X higher throughput and significantly lower latency for read workloads.
  12. ScyllaDB 2026.2: DynamoDB Streams and Vector Search, Trie Indexes, and Strongly Consistent TablesIntroduces DynamoDB compatible API (Alternator) enhancements, including DynamoDB Streams compatibility (now GA) and a Vector Search extension. Discusses performance improvements with Trie index as the default and connection storm mitigation. Mentions experimental features like Strongly Consistent Tables and vNode-to-Tablet online migration.
  13. Automate ScyllaDB X Cloud Clusters with TerraformThis post introduces the ScyllaDB Cloud Terraform provider's support for ScyllaDB X Cloud. It details how to provision and manage elastic, autoscaling ScyllaDB clusters using infrastructure-as-code. The post explains the X Cloud architecture's tablet-based approach to scaling and outlines key scaling parameters configurable via Terraform, such as instance families, storage policies (min_gb, target_utilization), and vCPU policies (min). It provides a concrete Terraform resource example for an X Cloud cluster.
  14. ScyllaDB Operator 1.21 Release — with Oracle Kubernetes Engine (OKE) SupportThe ScyllaDB Operator 1.21 release adds support for Oracle Kubernetes Engine (OKE), including comprehensive documentation and an automated setup script for infrastructure provisioning. It also introduces ECDSA as an alternative key type for TLS certificates, configurable via flags, and makes the Prometheus Operator an optional dependency.
  15. Using Salting to Lower Latency for Large Blobs in ScyllaDBIntroduces a modified salting technique for handling large blobs in ScyllaDB, demonstrating a 22x reduction in P99 write latency for 60MB blobs compared to default behavior. The technique involves splitting blobs into smaller chunks for parallel processing to mitigate memory saturation and head-of-line blocking issues.
  16. ScyllaDB Elastic Scaling in Action [Demo]This post demonstrates the elastic scaling capabilities of ScyllaDB X Cloud in action, showcasing a 10x scale-out from 100K ops/sec to 1M ops/sec in approximately 23 minutes while under load. It highlights the automatic tablet streaming and rebalancing process, the ability to maintain single-digit millisecond P99 latency during the scale-up, and the subsequent rapid scale-down to baseline load. The demo illustrates how the system automatically sizes nodes based on throughput and schema complexity and how new nodes begin serving requests before rebalancing is complete.
  17. Apache Cassandra Performance Tuning: What We LearnedThis post consolidates learnings from tuning Apache Cassandra for performance, emphasizing the need for deep understanding of configuration parameters and the architectural differences between ScyllaDB and Cassandra. It highlights that Cassandra's configuration comments can be misleading and that tuning requires environment-specific adjustments. The post also points to ScyllaDB's own performance comparison methodologies and resources.
  18. “Key-Value” is Misleading. Access Patterns are Key.This post elaborates on the importance of access patterns in database selection and data modeling, contrasting different data models (Key-Value, Wide-Column) and their implications for performance, I/O costs, and infrastructure spend. It highlights how ScyllaDB's architecture can accommodate various levels of access pattern depth, from simple key-value stores to more complex wide-column structures, and emphasizes that understanding application read/write behavior is crucial for optimal performance and cost-efficiency.