Global Distributed Databases

7/8/2026

Introduces Meerkat, an experimental consensus service built on the QuePaxa algorithm, designed for global control-plane state management. It addresses limitations of Raft in wide-area networks by allowing all replicas to perform writes and progress without timeouts. Meerkat aims to provide strong consistency (linearizability) and fault tolerance (availability as long as a majority of replicas are alive and can communicate) for applications like key-value stores and leasing systems.

5/14/2026

Identified and resolved a performance bottleneck in ClickHouse's query planning phase caused by increased data part count after implementing per-namespace retention. Developed and contributed three patches upstream: 1. Replaced exclusive locks with shared locks for read operations on the parts list. 2. Deferred vector copying by creating a shared copy of the parts list for read-only operations. 3. Implemented binary search for more efficient part selection.

4/16/2026

Introduces Artifacts, a new distributed, versioned filesystem built for agents, which speaks Git. Artifacts leverages Durable Objects and a custom Zig-based Git server (compiled to Wasm) to provide scalable, programmatic Git repositories. It supports creating repositories on the fly, importing existing repositories, forking, and time-travel. It also introduces ArtifactFS, an open-source filesystem driver for mounting large Git repos quickly by hydrating file contents on demand.

4/16/2026

This post announces the deeper integration of PlanetScale databases with Cloudflare Workers. Key technical contributions include: 1. Direct creation and billing of PlanetScale Postgres and MySQL databases from the Cloudflare dashboard and API, consolidating billing to Cloudflare accounts. 2. Enhanced integration with Hyperdrive, Cloudflare's database connectivity service, for managing connection pools and query caching. 3. The ability to configure explicit placement hints for Workers to execute closer to PlanetScale databases, reducing latency. 4. The use of `pg` client for Node.js to interact with PlanetScale Postgres databases via Workers bindings.

2/21/2026

This post details a significant outage caused by a bug in the BYOIP service's configuration management, specifically within the Addressing API. A change intended to automate the removal of prefixes for deletion was misconfigured, leading to the unintentional withdrawal of approximately 1,100 BYOIP prefixes. The incident highlights issues with staging environment data, incomplete test coverage for task-runner initiated changes, and the lack of a robust rollback mechanism for operational state. Recovery involved manual intervention for some customers due to removed service bindings and a software bug affecting state propagation to the edge. The post outlines steps to prevent future outages, including improvements to the Addressing API, health-mediated deployments, and more comprehensive testing.

12/18/2025

This post introduces support for aggregation queries (GROUP BY, SUM, COUNT, AVG, etc.) in R2 SQL, Cloudflare's serverless analytics query engine for data stored in R2 Data Catalog. It details two new techniques for executing these queries: scatter-gather aggregations for queries without HAVING or ORDER BY clauses, and shuffling aggregations for queries that require sorting or filtering based on aggregate results. Shuffling involves deterministic hash partitioning and a synchronization barrier to colocate data for specific groups before final aggregation and local finalization on worker nodes, followed by a k-way merge on the coordinator.

11/7/2025

Introduced a self-serve BYOIP API that automates the onboarding and setup of customer IP prefixes. This process leverages RPKI (specifically ROA objects) and IRR/rDNS record validation with unique tokens to verify ownership and intent, replacing the manual LOA process. The API currently supports prefixes originated from Cloudflare's ASN (AS13335) and aims to empower customers with greater control over their IP address management on the Cloudflare network.

9/25/2025

This post announces the Cloudflare Data Platform, a new integrated offering for analytical data. It comprises three key components: Cloudflare Pipelines, which uses Arroyo for stream processing to ingest and transform events into Iceberg or Parquet formats on R2; R2 Data Catalog, which now includes automatic compaction for performance optimization of Iceberg tables; and R2 SQL, a new distributed SQL engine for querying data stored in R2. The platform leverages Apache Iceberg as the open table format and Cloudflare's global network and R2 object storage, emphasizing zero egress fees and interoperability with other query engines.

9/25/2025

This post announces a partnership with PlanetScale, enabling direct connections from Cloudflare Workers to PlanetScale's MySQL and PostgreSQL databases. It highlights the integration with Cloudflare's Hyperdrive service, which provides connection pooling, query caching, and optimized routing for low-latency database access from the edge. The integration simplifies credential management by allowing account linking and offers one-click password rotation. The post details how Hyperdrive's Placement system determines optimal connection pool locations and resolves connection setups within the client to minimize roundtrips.

9/25/2025

Introduced R2 SQL, a serverless query engine for R2 object storage that leverages Apache Iceberg for data organization. Designed a two-phase architecture: a Query Planner for intelligent data pruning using partition-level and column-level statistics from Iceberg metadata, and a Query Execution system for massively parallel processing across Cloudflare's global network. Implemented a streamed planning pipeline for early execution and a 'stop early' strategy based on ordered results and metadata to minimize data read.

8/8/2025

Redesigned Workers KV to use an internal distributed database for small objects and R2 for larger objects, eliminating reliance on third-party object storage providers. Implemented KV Storage Proxy (KVSP) to bridge HTTP-based Workers KV requests with the database's binary protocol and shard namespaces. Restored dual-provider write capabilities for enhanced durability and introduced background reconciliation for failed writes.

7/17/2025

This post details the evolution of Cloudflare's Quicksilver globally distributed key-value store to version 2. It addresses scalability challenges faced by V1 and V1.5 by introducing a new architecture that separates storage replicas from caching proxies. Key innovations include identifying and offloading 'cold keys' to dedicated storage servers, implementing a relay mechanism to manage connections from caching proxies to replicas, and introducing reactive prefetching of cache misses to improve cache hit rates to 99.9%. The post also discusses the need for further cache hit rate improvements for latency-sensitive instances like 'dnsv2' and the reintroduction of data center-wide sharded caches before accessing replicas.

7/10/2025

Introduced Quicksilver v1.5 with replica and proxy roles to optimize disk space by caching hot keys on proxies instead of replicating the full dataset everywhere. Implemented persistent caching using RocksDB and compaction filters for eviction. Addressed consistency challenges introduced by asynchronous replication in the proxy/replica model by implementing Multiversion Concurrency Control (MVCC) to maintain sequential consistency.

7/8/2025

This post details the adoption of TimescaleDB (a PostgreSQL extension) for the analytics and reporting needs of Cloudflare's Digital Experience Monitoring (DEX) product suite, specifically for Zero Trust. It explains the decision-making process, contrasting it with ClickHouse and highlighting the benefits of a simpler, PostgreSQL-based architecture for the initial MVP. The post dives into the technical implementation of using PostgreSQL for storing device state logs, including table schema design, index optimization (multi-column B-tree indexes with specific column ordering for time-series data), and the use of UPSERT for deduplication. It also discusses the trade-offs and design decisions made to prioritize shipping fast with a simpler architecture.

4/10/2025

Introduced R2 Data Catalog, a managed Apache Iceberg catalog integrated directly into Cloudflare R2 buckets. This provides database-like capabilities (ACID transactions, schema evolution, efficient querying) for data stored in object storage, exposing a standard Iceberg REST catalog interface for compatibility with engines like PyIceberg, Snowflake, and Spark. The offering has zero egress fees.

4/10/2025

Introduced the D1 Sessions API to enable sequential consistency for read-after-write operations across globally distributed read replicas. The Sessions API manages session bookmarks to ensure that read queries on replicas are at least as up-to-date as the last observed write, guaranteeing 'read your own writes' and 'writes follow reads' properties even when client requests are routed to different database instances.

4/8/2025

This post introduces support for MySQL and MariaDB databases within Cloudflare Workers and Hyperdrive. It details the technical challenges of integrating traditional MySQL drivers (mysql and mysql2) with the Workers runtime, including overcoming Node.js dependency limitations and the use of `eval()`. The post also explains how Hyperdrive has been adapted to parse the MySQL wire protocol, handle its authentication plugins and capability flags, and provide global connection pooling and caching for MySQL databases, enabling developers to build global applications on Workers connected to these relational databases.

4/7/2025

This post announces the acquisition of Outerbase by Cloudflare, which will integrate Outerbase's data management and developer experience features into Cloudflare's database offerings, specifically D1 and Durable Objects. Key integrations include a data explorer, query runner, automatic REST APIs, row-level security, WebSocket support for streaming queries, and pre- and post-query hooks for D1. The acquisition aims to simplify database management and enhance the developer experience for users of Cloudflare's data services.

3/25/2025

This post details an incident where a human error in credential rotation for the R2 Gateway service led to a temporary authentication failure with the storage infrastructure. The error involved inadvertently deploying new credentials to a development instance instead of production due to omitting the '--env' parameter in Wrangler commands. This incident highlights the importance of proper visibility into credential usage and automated checks to prevent such misdeployments. The resolution involved identifying the root cause and deploying the correct credentials to the production environment.

2/7/2025

This post details a significant incident where the R2 Gateway service was inadvertently disabled due to human error and insufficient validation safeguards during an abuse remediation process. It describes the impact on R2 and dependent services (Stream, Images, Cache Reserve, Vectorize, Log Delivery), the incident timeline, the root cause (human error and validation gap), and the remediation steps. It also highlights the R2 architecture's separation of concerns and the secondary impact on Durable Objects due to client reconnection spikes.

10/22/2024

This post details the architecture and implementation of Vectorize, a globally distributed vector database built on Cloudflare's Developer Platform. It explains the core components, query processing architecture, indexing strategies (IVF and PQ), approximate nearest neighbor search with result refinement, and the eventual consistency model achieved through snapshot versioning and a WAL implemented with SQLite in Durable Objects. It highlights the use of R2 for storage and Cloudflare's Cache for I/O optimization.

9/26/2024

Introduces the ability to use SQLite directly within Durable Objects, providing a full SQL query interface with tables and indexes. This enables zero-latency storage by running the database as a local library embedded within the application thread. It also details the implementation of synchronous queries with 'Output Gates' to maintain high throughput and low latency for writes while ensuring durability.

6/28/2024

Added support for Postgres protocol-level named prepared statements in Hyperdrive. This involved deep dives into the Postgres Message Protocol, specifically the extended query flow involving Parse, Bind, Execute, and Sync messages. Challenges included buffering messages for caching, managing session scope of prepared statements within a pooled connection environment, and implementing logic to handle four scenarios: Parse with cache hit, Parse with cache miss, Reuse with cache hit, and Reuse with cache miss. Utilized Rust data structures like HashMap, LruCaches, and VecDeque for efficient message management.

4/3/2024

This post details the integration of Prisma ORM with Cloudflare Workers and D1, enabling developers to use Prisma's type-safe client API for data interactions. It outlines the setup process, including configuring `wrangler.toml`, installing necessary packages (`@prisma/client`, `@prisma/adapter-d1`), initializing Prisma, defining a schema with the `driverAdapters` preview feature, creating migrations, and integrating the Prisma Client with D1 in a Cloudflare Worker.

4/1/2024

Introduces asynchronous read replication for D1 to support globally-distributed applications. Details the concept of read replicas, replica lag, and consistency models (read committed, snapshot isolation, serializable). Explains how D1 currently provides snapshot isolation via Durable Objects and the challenges of routing to consistent replicas. Proposes a new D1 Sessions API that leverages Lamport timestamps and a delay mechanism on replicas to achieve sequential consistency, enabling read-heavy workloads to utilize read replicas for lower latency and increased throughput.

4/1/2024

This post announces the General Availability of D1, Cloudflare's serverless SQL database, with support for 10GB databases, 50,000 databases per account, data export, and enhanced query debugging ('D1 Insights'). It also introduces upcoming features like global read replication and a new Session API for maintaining consistency across replicas. Hyperdrive, which makes existing databases feel distributed, is also GA, offering optimized routing, connection pooling, and query caching, with future support for MySQL and private network connections. Hyperdrive is free for developers on a Workers Paid plan. Cloudflare Queues now supports pull-based consumers, allowing any HTTP-speaking client to pull and acknowledge messages, and introduces new message delivery controls.

9/28/2023

Introduced Hyperdrive, a new service that acts as a global proxy and cache for existing databases. Hyperdrive maintains regional connection pools to reduce latency from establishing new database connections and caches popular read queries to avoid hitting the origin database. It supports PostgreSQL and is expanding to MySQL, and integrates with existing database drivers and ORMs by providing a modified connection string.

9/28/2023

This post announces the open beta of D1, significantly increasing per-database storage limits to 2GB and the number of databases per account to 50,000. It highlights the introduction of 'Time Travel' for point-in-time database recovery (enabled by default and free) and the shift to a row-based pricing model. The post also details how these changes enable production-scale applications, database-per-user isolation, and improved cost management.

5/19/2023

This post announces a major update to Cloudflare's D1 database, introducing a new storage backend that significantly improves performance (up to 20x faster reads, 6.8-11x faster writes) and consistency. It also introduces 'Time Travel' for point-in-time recovery (restoring to any minute within the last 30 days) and 'Location Hints' to influence global leader placement. Developer experience enhancements include a new dashboard console for issuing queries, formal support for JSON functions, and updated documentation. Pricing details are also shared, outlining an always-free tier, included usage in the Workers subscription, and pay-as-you-go for reads, writes, and storage, with no additional cost for global read replication.

5/16/2023

Introduced 'Database Integrations' for Cloudflare Workers, enabling seamless connection to third-party databases (Neon, PlanetScale, Supabase) via HTTP. This feature simplifies discovery and authentication through an OAuth2 flow, storing credentials as encrypted environment variables. It also announces upcoming support for more databases and an integrations platform for third-party providers, along with Wrangler CLI integration and account-level environment variables.

5/16/2023

This post details the integration of Cloudflare R2 object storage with MosaicML's StreamingDataset and Composer libraries to enable efficient training of large language models (LLMs). It demonstrates how to stream training data and save/load model checkpoints directly to/from R2, leveraging R2's zero-egress pricing to facilitate multi-cloud GPU utilization. Code examples are provided for writing data to R2 using `awscli` and reading data with `StreamingDataset`, as well as saving checkpoints with `Composer`. The post highlights the flexibility and cost savings achieved by decoupling data storage from compute.

5/16/2023

This post announces a partnership between Cloudflare R2 and Snowflake, enabling users to query data stored in R2 data lakes using Snowflake and load data from R2 into Snowflake. It details the process of generating R2 API tokens and creating external stages in Snowflake using S3-compatible endpoints. It also provides SQL examples for loading data with `COPY INTO` and querying external tables.

11/16/2022

Introduces the open alpha for D1, a serverless relational database built on Cloudflare Workers. Details include: dashboard integration for database creation and management, Wrangler CLI for executing SQL files and managing migrations, D1 binding for Workers and Pages Functions, and highlights community-contributed tooling like d1-orm, workers-qb, d1-console, and a Kysely adapter. It also outlines current limitations such as single database instance location, concurrency queuing, and backup blocking.

11/16/2022

This post introduces Cloudflare D1, a serverless relational database service, and demonstrates its integration with Cloudflare Workers to add dynamic features like comments to static websites. It details the process of creating a D1 database, configuring it with wrangler, defining a schema, and building API endpoints for retrieving and creating comments using SQL queries executed via the `DB` binding in Workers. The post highlights D1's ability to provide a developer-friendly, SQL-based data layer for serverless applications, bridging the gap between static site generators and dynamic data.

11/15/2022

This post details the integration of Neon.tech's serverless PostgreSQL driver (`@neondatabase/serverless`) with Cloudflare Workers. It explains how a WebSocket proxy and a client library that shims Node.js features (including TLS via WebAssembly) enable Workers to connect to PostgreSQL databases, overcoming the lack of raw TCP support in Workers. It also demonstrates using PostGIS for geospatial queries within a Worker.

9/27/2022

Introduced the concept of D1 stored procedures to enable JavaScript transactions that interleave SQL and JS code. This is achieved via a new `db.transaction()` API within a dedicated procedures file, allowing for atomic operations that execute on the primary D1 instance to maintain consistency. This addresses the challenge of blocking the entire database with long-running transactions that involve multiple round trips between Workers and D1.

9/21/2022

This post announces the General Availability of R2, Cloudflare's object storage service. It highlights key features including S3 compatibility with code examples, automatic region selection, deep integration with Cloudflare Workers (with a code example for a REST API), presigned URLs (with a code example), public buckets, and a transparent pricing model. It also outlines future plans for object lifecycles, jurisdictional restrictions, and live migration without downtime. The post also mentions that Cloudflare Images is now powered by R2.

8/26/2022

This post details the implementation of gateway-level concurrency throttling in a forked version of PgBouncer to address performance isolation issues in multi-tenant Postgres clusters. It introduces runtime commands and configuration options to dynamically set connection limits per user and pool, and explores future solutions like TCP Vegas-inspired congestion avoidance and tenant resource quotas for more robust performance management.

8/26/2022

This post details Cloudflare's open-sourcing of its fork of PgBouncer. Key contributions include fixing an authentication bug that prevented user-level connection pool limits from working with HBA authentication, and implementing new features for enforcing per-user and per-pool connection limits at runtime. These changes aim to improve tenant performance isolation in their multitenant Postgres environments.

5/11/2022

This post announces the launch of D1, Cloudflare's first SQL database, built on SQLite. It highlights D1's serverless nature, integration with Cloudflare Workers, and developer experience. Key features introduced include read replication for global distribution, batching for efficient multi-statement execution, embedded compute for co-located logic, and automatic backups to R2. The post also details data import/export capabilities and the pricing model.

11/15/2021

Introduces the initial capability for connecting Cloudflare Workers to relational databases (Postgres, MySQL) by leveraging cloudflared for secure tunnels and a shim-layer to adapt socket APIs to WebSockets. This enables developers to query existing databases from Workers, addressing a key missing piece in the Workers compute platform. It outlines future plans for native TCP support, connection pooling, and global data caching.

11/15/2021

This post announces the general availability and production readiness of Durable Objects. It details their core functionality: unique, single-threaded instances processing messages in-order with strongly consistent key-value storage. It highlights their benefits for global scale, concurrency, and simplifying complex applications by eliminating the need for traditional databases and their associated management. The post also outlines improvements made since early access, including enhanced reliability, performance, powering other Cloudflare products (R2, Waiting Room), customer adoption (gaming backends, status pages, collaborative whiteboards), operational improvements for high request volumes, Jurisdictional Restrictions, caching for storage operations, and simplified deployment/debugging tools. Future plans include replication for multi-threaded access, guaranteed callbacks to wake objects, and better geo-distribution.

9/28/2021

Introduced Cloudflare R2 Storage, a new object storage service with S3 API compatibility and zero egress fees. R2 is priced at $0.015 per GB/month for storage and offers a cost-effective per-operation charge above a certain threshold. It features automatic intelligent tiering for performance and cost optimization. A migration service is provided to facilitate data transfer from S3-compatible storage. R2 guarantees eleven 9s of annual durability and is designed for high availability. It integrates with Cloudflare Workers for dynamic data transformation and manipulation.

8/4/2021

This post introduces a tutorial for connecting to Postgres databases within Cloudflare Workers functions using PostgREST. It demonstrates how to build a REST API on the edge that communicates directly with a Postgres database. It also highlights the use of Cloudflare Tunnel for securely exposing the PostgREST endpoint and the `postgrest-js` library for interacting with PostgREST from Workers. The post discusses scaling strategies for Postgres applications by leveraging Workers' architecture for caching and performance optimization.

8/3/2021

This post details significant improvements to the Durable Objects runtime that make them correct by default and faster. Previously, applications using Durable Objects' key/value storage API could suffer from subtle race conditions due to the interleaving of asynchronous operations (awaits) and slow disk writes. The new runtime processes writes asynchronously, ensuring data durability without blocking application logic, thus eliminating race conditions and improving performance by avoiding the need for explicit transactions or in-memory caching for correctness.

7/23/2021

This post analyzes and criticizes AWS's egress bandwidth pricing model, highlighting its significant markups compared to wholesale bandwidth costs and the industry trend of waiving or discounting egress fees through initiatives like the Bandwidth Alliance. It contrasts AWS's 'stocks' pricing model with the industry's 'flows' model and argues that AWS's pricing strategy locks customers into their cloud. The post also touches upon the asymmetry of AWS charging for egress but not ingress, which is contrary to how wholesale bandwidth is priced. This contributes to the understanding of Cloudflare's value proposition in offering cost-effective and transparent bandwidth solutions.

7/23/2021

This post introduces the Bandwidth Alliance as a mechanism to reduce egress fees for data transfer between Cloudflare and its partners. It explains how Cloudflare's global network and private interconnects enable low-cost data transfer to partner data centers. The post highlights customer benefits, including cost savings and increased choice, and mentions specific case studies and the integration with Microsoft Azure for discounted egress.

7/6/2021

This post details the integration of Cloudflare's network with Microsoft Azure's Data Transfer Routing Preference service. It explains how Cloudflare's extensive peering and private interconnections with Azure enable customers to route traffic between Cloudflare's edge and Azure origins with reduced costs and improved performance. The post outlines the technical steps for customers to enable this feature in their Azure and Cloudflare dashboards, highlighting the benefits of direct, private interconnections for egress traffic.

4/16/2021

This post announces partnerships with Macrometa and Fauna to provide edge-first database solutions for Cloudflare Workers. Macrometa offers a globally replicated NoSQL database with integrated search, pub/sub, and stream processing, providing low latency for data calls between Workers and Macrometa. Fauna offers a global transactional database as a data API, supporting GraphQL and custom business logic at the edge, complementing Workers KV and Durable Objects with strong consistency and queryable persistence.

3/31/2021

This post announces the open beta of Durable Objects, a new paradigm for stateful serverless computing. Durable Objects provide coordination across multiple Workers and strongly consistent edge storage. Each Durable Object is defined by a JavaScript class and a unique ID, ensuring only one instance runs globally. This allows for coordination across multiple requests and enables Workers to make requests to downstream APIs and store data with a strongly consistent storage API. The post details pricing, comparing it favorably to AWS Lambda and DynamoDB, and highlights features like in-memory caching, multi-region replication by default, and strong consistency at no additional charge. It also mentions recent feature additions like smarter initial placement for user-named Objects, jurisdictional restrictions, and Wrangler support.

12/12/2020

Introduced Jurisdictional Restrictions for Durable Objects, allowing developers to specify a geographical region (e.g., 'eu') for data storage and processing when generating Durable Object IDs. This ensures compliance with data localization regulations without requiring developers to manage separate infrastructure. The implementation encodes jurisdictional metadata within the Durable Object ID, mapping it to specific Cloudflare data centers within the designated region. This feature is currently not supported for user-generated IDs.

9/28/2020

Introduced Durable Objects, a new serverless primitive for stateful applications on Cloudflare Workers. Durable Objects provide globally unique, durable, and strongly consistent storage co-located with the object instance. They enable real-time coordination between clients via WebSockets and other mechanisms, allowing for the development of fully edge-hosted applications without a traditional origin server. The post details the concepts of 'Objects', 'Unique', and 'Durable', and provides examples for an atomic counter and a chat application.

6/9/2020

Introduces UtahFS, a proof-of-concept encrypted file storage system. UtahFS utilizes object storage for the underlying storage layer, breaks files into 32KB blocks connected by a skip list for efficient partial file access, and employs AES-GCM for confidentiality and Merkle Trees for integrity. It also incorporates Path ORAM to hide access patterns from the storage provider. The system is open-source and documented but not used in production at Cloudflare.

3/30/2020

This post details the replacement of Kyoto Tycoon (KT) with a new system called Quicksilver for configuration distribution at Cloudflare's internet scale. It highlights the limitations of KT, including exclusive write lock issues that degraded read latency and data corruption problems due to disabled fsync for performance. The post describes the architecture of the previous KT-based system and the new Quicksilver system, emphasizing the need for a fast, reliable, and scalable solution to distribute configuration changes to 200 cities within seconds. The technical challenges encountered with KT, such as poor concurrent access performance and operational difficulties in managing thousands of instances, are thoroughly explained. The post serves as a case study in outgrowing a datastore and building a replacement from scratch to meet evolving infrastructure demands.

11/6/2019

Introduced new APIs for Workers KV: bulk upload/delete of key/value pairs, and listing keys with pagination (cursor) and prefix filtering. Increased the maximum value size limit from 2MB to 10MB and the number of namespaces per account from 20 to 100. Added a KV browser to the dashboard for managing keys and values. Enhanced Wrangler CLI integration for KV namespace creation and key management.

5/21/2019

Introduced Workers KV as a general availability product, a highly distributed, eventually consistent key-value store for Cloudflare's global edge. It guarantees Availability and Partition Tolerance, enabling ultra-low latency reads powered by caching technology, with global consistency achievable in under 60 seconds. New features include bulk writes for efficient data import (up to 10,000 pairs in a single PUT request) and expiring keys using `expiration` and `expirationTtl` options for write operations.

11/15/2018

This post introduces the Bandwidth Alliance, a program offering zero-egress rates for data transfer between Cloudflare and its partners. It details a solution design methodology for selecting storage providers based on customer needs (zero-egress, cost, read/write rates, volume, object size, S3 API compatibility, security) and categorizes providers based on their fit. It highlights the importance of aligning provider economics with specific use cases, particularly regarding read request pricing.

9/28/2018

Introduced Workers KV, a highly distributed, eventually-consistent, key-value store designed for global low-latency reads. It allows storing up to a billion keys and values with ultra-low latency anywhere in the world, enabling applications to achieve performance comparable to static content delivery. Use cases include API gateways, dynamic data serving, configuration management, and inter-system integration.

4/9/2015

This post details the scaling of Cloudflare's PostgreSQL-based analytics infrastructure using CitusDB. It describes the log processing pipeline, the rationale for choosing Go and Kafka, and the specific criteria for selecting CitusDB (performance, PostgreSQL compatibility, scaling, high availability, and BI query support). The CitusDB architecture, including its primary node and worker nodes for parallel query execution, is explained. The post also highlights the benefits of PostgreSQL extensions like HyperLogLog and how CitusDB scales them across multiple servers.

12/22/2014

This post details the implementation of mutually authenticated TLS tunnels for Kyoto Tycoon, a distributed key-value store used for replicating sensitive data (session ticket keys, DNS data) between Cloudflare's data centers. This enhancement protects data in transit over the public internet by ensuring both the client and server authenticate each other using TLS client authentication, preventing unauthorized access and modification. The changes, implemented using OpenSSL and an internal CA, have been contributed back to the open-source Kyoto Tycoon project.

7/8/2013

Introduced a Foreign Data Wrapper (FDW) for Kyoto Tycoon (KT) that leverages PostgreSQL 9.3's writable FDW capabilities. This allows data in KT to be represented as a PostgreSQL table, enabling the use of database triggers for data synchronization between PostgreSQL and KT. This integration reduces latency and simplifies synchronization logic. It also translates PostgreSQL transactions into KT transactions, exporting ACID guarantees to KT and allowing transactional operations using SQL commands.

6/30/2012

This post discusses the AWS outage and its impact on services relying on IaaS. It differentiates between SaaS, PaaS, and IaaS, and highlights the challenges of data synchronization and scaling in the Data & Application layer. It then introduces the Front End layer as a candidate for distributed scaling, explaining how Cloudflare's platform acts as a scalable front-end layer that can be provisioned with a DNS change, offering acceleration and protection without application layer changes. It uses the AWS outage as a case study to illustrate the resilience of Cloudflare's front-end approach compared to IaaS.