TechPrufer
Get started
Cloudflare
Cloudflare logoCloudflare · Feature trail

Global Distributed Databases

65 posts · Jun 2012 – Jul 2026

What this trail covers

Cloudflare Workers now supports direct integration with PostgreSQL databases, enabling developers to build serverless applications with familiar relational database backends. This integration leverages Hyperdrive for global connection pooling and caching, and PostgREST for exposing PostgreSQL functionality as a REST API. Hyperdrive has evolved to support Postgres protocol-level named prepared statements, improving performance and developer experience by caching query execution plans and handling session scope complexities.

All Cloudflare trails
  1. Introducing Meerkat- an experiment in global consensusIntroduces Meerkat, an experimental consensus service built on the QuePaxa algorithm, designed for global control-plane state management. It addresses limitations of Raft in wide-area networks by allowing all replicas to perform writes and progress without timeouts. Meerkat aims to provide strong consistency (linearizability) and fault tolerance (availability as long as a majority of replicas are alive and can communicate) for applications like key-value stores and leasing systems.
  2. Our billing pipeline was suddenly slow. The culprit was a hidden bottleneck in ClickHouseIdentified and resolved a performance bottleneck in ClickHouse's query planning phase caused by increased data part count after implementing per-namespace retention. Developed and contributed three patches upstream: 1. Replaced exclusive locks with shared locks for read operations on the parts list. 2. Deferred vector copying by creating a shared copy of the parts list for read-only operations. 3. Implemented binary search for more efficient part selection.
  3. Artifacts: versioned storage that speaks GitIntroduces Artifacts, a new distributed, versioned filesystem built for agents, which speaks Git. Artifacts leverages Durable Objects and a custom Zig-based Git server (compiled to Wasm) to provide scalable, programmatic Git repositories. It supports creating repositories on the fly, importing existing repositories, forking, and time-travel. It also introduces ArtifactFS, an open-source filesystem driver for mounting large Git repos quickly by hydrating file contents on demand.
  4. Deploy Postgres and MySQL databases with PlanetScale + WorkersThis post announces a partnership with PlanetScale, enabling Cloudflare Workers to directly access PlanetScale Postgres and MySQL databases. It details the integration via Cloudflare's dashboard and API, with future billing to Cloudflare accounts. The post highlights the use of Hyperdrive for connection pooling and query caching, and the ability to configure explicit placement hints for Workers to run closer to the databases, reducing latency. It also mentions PlanetScale's developer experience features like query insights and branching.
  5. Cloudflare outage on February 20, 2026This post details a significant outage caused by a bug in the BYOIP service's configuration management, specifically within the Addressing API. A change intended to automate the removal of prefixes for deletion was misconfigured, leading to the unintentional withdrawal of approximately 1,100 BYOIP prefixes. The incident highlights issues with staging environment data, incomplete test coverage for task-runner initiated changes, and the lack of a robust rollback mechanism for operational state. Recovery involved manual intervention for some customers due to removed service bindings and a software bug affecting state propagation to the edge. The post outlines steps to prevent future outages, including improvements to the Addressing API, health-mediated deployments, and more comprehensive testing.
  7. Announcing support for GROUP BY, SUM, and other aggregation queries in R2 SQLThis post introduces support for aggregation queries (GROUP BY, SUM, COUNT, AVG, etc.) in R2 SQL, Cloudflare's serverless analytics query engine for data stored in R2 Data Catalog. It details two new techniques for executing these queries: scatter-gather aggregations for queries without HAVING or ORDER BY clauses, and shuffling aggregations for queries that require sorting or filtering based on aggregate results. Shuffling involves deterministic hash partitioning and a synchronization barrier to colocate data for specific groups before final aggregation and local finalization on worker nodes, followed by a k-way merge on the coordinator.
  8. DIY BYOIP- a new way to Bring Your Own IP prefixes to CloudflareIntroduced a self-serve BYOIP API that automates the onboarding and setup of customer IP prefixes. This process leverages RPKI (specifically ROA objects) and IRR/rDNS record validation with unique tokens to verify ownership and intent, replacing the manual LOA process. The API currently supports prefixes originated from Cloudflare's ASN (AS13335) and aims to empower customers with greater control over their IP address management on the Cloudflare network.
  9. Announcing the Cloudflare Data Platform: ingest, store, and query your data directly on CloudflareThis post announces the Cloudflare Data Platform, a new integrated offering for analytical data. It comprises three key components: Cloudflare Pipelines, which uses Arroyo for stream processing to ingest and transform events into Iceberg or Parquet formats on R2; R2 Data Catalog, which now includes automatic compaction for performance optimization of Iceberg tables; and R2 SQL, a new distributed SQL engine for querying data stored in R2. The platform leverages Apache Iceberg as the open table format and Cloudflare's global network and R2 object storage, emphasizing zero egress fees and interoperability with other query engines.
  10. Partnering to make full-stack fast: deploy PlanetScale databases directly from WorkersThis post announces the direct integration of PlanetScale PostgreSQL and MySQL databases with Cloudflare Workers, facilitated by Hyperdrive. It details how users can connect their PlanetScale accounts to Cloudflare for seamless database access from Workers, eliminating manual credential management and leveraging Hyperdrive's connection pooling and query caching for performance optimization. The integration also allows for viewing Hyperdrive configurations from the PlanetScale dashboard.
  11. R2 SQL: a deep dive into our new distributed query engineIntroduced R2 SQL, a serverless query engine for R2 object storage that leverages Apache Iceberg for data organization. Designed a two-phase architecture: a Query Planner for intelligent data pruning using partition-level and column-level statistics from Iceberg metadata, and a Query Execution system for massively parallel processing across Cloudflare's global network. Implemented a streamed planning pipeline for early execution and a 'stop early' strategy based on ordered results and metadata to minimize data read.
  12. Redesigning Workers KV for increased availability and faster performanceRedesigned Workers KV to use an internal distributed database (the same one powering R2 and Durable Objects) as its primary storage, replacing reliance on third-party object storage providers. Implemented a hybrid storage architecture where KV uses the distributed database for small objects and R2 for larger objects. Restored dual-provider write capabilities by racing writes to both the new internal database and a secondary provider (likely the previously used one), with background reconciliation for failed writes. Introduced KV Storage Proxy (KVSP) to bridge HTTP communication from SGW to the database clusters and stripe namespaces across clusters.
  13. Quicksilver v2: evolution of a globally distributed key-value store (Part 2)This post details the evolution of Cloudflare's Quicksilver globally distributed key-value store to version 2. It addresses scalability challenges faced by V1 and V1.5 by introducing a new architecture that separates storage replicas from caching proxies. Key innovations include identifying and offloading 'cold keys' to dedicated storage servers, implementing a relay mechanism to manage connections from caching proxies to replicas, and introducing reactive prefetching of cache misses to improve cache hit rates to 99.9%. The post also discusses the need for further cache hit rate improvements for latency-sensitive instances like 'dnsv2' and the reintroduction of data center-wide sharded caches before accessing replicas.
  14. Quicksilver v2: evolution of a globally distributed key-value store (Part 1)Introduced Quicksilver v1.5, a proxy-replica architecture to reduce disk space by caching hot keys on disk (RocksDB) instead of replicating the full dataset everywhere. Implemented persistent caching with RocksDB compaction filters and LRU-like tracking for evictions. Addressed consistency challenges with asynchronous replication by introducing MVCC to maintain sequential consistency, storing historical versions of keys with replication indices.
  15. How TimescaleDB helped us scale analytics and reportingThis post details the adoption of TimescaleDB (a PostgreSQL extension) for the analytics and reporting needs of Cloudflare's Digital Experience Monitoring (DEX) product suite, specifically for Zero Trust. It explains the decision-making process, contrasting it with ClickHouse and highlighting the benefits of a simpler, PostgreSQL-based architecture for the initial MVP. The post dives into the technical implementation of using PostgreSQL for storing device state logs, including table schema design, index optimization (multi-column B-tree indexes with specific column ordering for time-series data), and the use of UPSERT for deduplication. It also discusses the trade-offs and design decisions made to prioritize shipping fast with a simpler architecture.
  16. R2 Data Catalog: Managed Apache Iceberg tables with zero egress feesIntroduced R2 Data Catalog, a managed Apache Iceberg catalog integrated directly into Cloudflare R2 buckets. This provides database-like capabilities (ACID transactions, schema evolution, efficient querying) for data stored in object storage, exposing a standard Iceberg REST catalog interface for compatibility with engines like PyIceberg, Snowflake, and Spark. The offering has zero egress fees.
  17. Sequential consistency without borders: How D1 implements global read replicationIntroduces D1 read replication and the new Sessions API. The Sessions API enables sequential consistency for queries within a session, even when routed to different replicas. It uses bookmarks to maintain session state across requests, ensuring properties like 'read my own writes' and 'writes follow reads'. The post details the implementation of sessions, bookmarking, and the benefits for read-heavy applications.
  18. Build global MySQL apps using Cloudflare Workers and HyperdriveThis post details the engineering effort to enable direct MySQL connectivity from Cloudflare Workers. It covers the challenges of Node.js compatibility for MySQL drivers (`mysql` and `mysql2`), including the need for polyfills for `net`, `stream`, `timers`, and the resolution of the `eval()` API limitation for `mysql2` by using static parsers. It also explains the integration of MySQL protocol support into Hyperdrive, a global connection pooling and caching solution, highlighting the adaptations required for MySQL's authentication plugins and capability flags. The post demonstrates how to configure Hyperdrive for MySQL databases hosted anywhere and provides code examples for connecting to MySQL from Workers using both `mysql` and `mysql2` drivers.
  19. Pools across the sea: how Hyperdrive speeds up access to databases and why we’re making it freeThis post announces that Hyperdrive is now available on the free plan of Cloudflare Workers. It details how Hyperdrive improves database access performance by reducing connection round trips and latency, illustrating this with a benchmark comparing direct connections to Hyperdrive-enabled connections. The post also explains Hyperdrive's design as a transaction-mode connection pooler and its role in solving the challenges of connecting serverless applications to traditional SQL databases.
  20. Cloudflare acquires Outerbase to expand database and agent developer experience capabilitiesThis post announces Cloudflare's acquisition of Outerbase, integrating its data management and visualization capabilities into Cloudflare's developer platform. Key contributions include adapting Outerbase's table viewer and query runner for D1 and Durable Objects, improving the getting started experience for Durable Objects, enhancing Workflow visualization, and introducing pre- and post-query hooks for D1. The acquisition also brings Starbase API features like WebSockets support to D1, expanding its REST API.
  21. Cloudflare incident on March 21, 2025This post details an incident where a human error in credential rotation for the R2 Gateway service led to a temporary authentication failure with the storage infrastructure. The error involved inadvertently deploying new credentials to a development instance instead of production due to omitting the '--env' parameter in Wrangler commands. This incident highlights the importance of proper visibility into credential usage and automated checks to prevent such misdeployments. The resolution involved identifying the root cause and deploying the correct credentials to the production environment.
  22. Cloudflare incident on February 6, 2025This post details a significant incident where the R2 Gateway service was inadvertently disabled due to human error and insufficient validation safeguards during an abuse remediation process. It describes the impact on R2 and dependent services (Stream, Images, Cache Reserve, Vectorize, Log Delivery), the incident timeline, the root cause (human error and validation gap), and the remediation steps. It also highlights the R2 architecture's separation of concerns and the secondary impact on Durable Objects due to client reconnection spikes.
  23. Migrating billions of records: moving our active DNS database while it’s in useThis post details the technical challenges and solutions involved in migrating billions of DNS records from a monolithic Postgres cluster (cfdb) to a new, partitioned database cluster (dnsdb). Key contributions include the design and implementation of a Change Data Capture and Transfer Service (CDCTS) to handle near real-time data migration, the logical decoupling of zone data from DNS records, the creation of a new internal DNS Records gRPC API for controlled access, and the partitioning of the _cf_rec table by zone_id. The migration process involved an initial data copy followed by continuous change replication, with a focus on bidirectional switchback capability and auditability.
  24. Building Vectorize, a distributed vector database, on Cloudflare’s Developer PlatformThis post details the architecture and implementation of Vectorize, a globally distributed vector database built on Cloudflare's Developer Platform. It explains the core components, query processing architecture, indexing strategies (IVF and PQ), approximate nearest neighbor search with result refinement, and the eventual consistency model achieved through snapshot versioning and a WAL implemented with SQLite in Durable Objects. It highlights the use of R2 for storage and Cloudflare's Cache for I/O optimization.
  25. Zero-latency SQLite storage in every Durable ObjectThis post introduces the integration of SQLite with Durable Objects (DOs), enabling zero-latency local storage access. It details how DOs run application code on the same thread as the data, eliminating network latency. The post explains the benefits of synchronous SQL queries for avoiding subtle bugs and introduces 'Output Gates' to ensure durability without sacrificing throughput or increasing latency. It also highlights how this architecture simplifies common patterns like N+1 query problems by allowing single, efficient queries.
  26. Supporting Postgres Named Prepared Statements in HyperdriveThis post details the implementation of named prepared statements for Postgres protocol within Cloudflare Hyperdrive. It explains the challenges of buffering Postgres messages for caching, managing session-scoped prepared statements across pooled connections, and the logic for handling Parse, Bind, Execute, and Sync messages. The solution involves in-memory storage of parsed statements and logic to serve cached results or proxy to origin connections.
  27. Improving Cloudflare Workers and D1 developer experience with Prisma ORMIntroduces native support for Cloudflare D1 databases within Cloudflare Workers using Prisma ORM. Details the setup process including `wrangler d1 create`, `wrangler.toml` configuration, installing Prisma dependencies (`@prisma/client`, `@prisma/adapter-d1`), initializing Prisma with `npx prisma init --datasource-provider sqlite`, enabling the `driverAdapters` preview feature, defining a `Visit` model, generating and applying migrations using `wrangler d1 migrations create` and `prisma migrate diff`, and re-generating the Prisma Client. Provides example code for a Worker to count and create visitor records using the Prisma Client.
  28. Building D1: a Global DatabaseThis post introduces the concept of asynchronous read replication for D1, explaining its benefits for global applications and detailing the challenges of maintaining consistency across replicas. It proposes a solution using a new Sessions API that leverages Lamport timestamps and routes queries to the primary for writes and to replicas (with delay) for reads to achieve sequential consistency. The post also contrasts D1's default snapshot isolation with the read committed model that can arise from inconsistent replica routing.
  29. Making state easy with D1 GA, Hyperdrive, Queues and Workers Analytics Engine updatesAnnounces General Availability (GA) for D1, Hyperdrive, and Workers Analytics Engine. Details D1's GA features (10GB databases, 50k per account, data export, D1 Insights) and future roadmap (global read replication, larger DBs, Time Travel, dynamic APIs). Introduces the D1 Session API for maintaining session-based consistency with read replicas. Hyperdrive is also GA, supporting popular drivers/ORMs, with future support for MySQL and private network connections via Cloudflare Tunnel/Magic WAN. Hyperdrive is free for Workers Paid plan users. Cloudflare Queues now supports pull-based consumers and new message delivery controls, allowing any HTTP-speaking client to pull messages and acknowledge them.
  30. Hyperdrive: making databases feel like they’re globalIntroduces Hyperdrive, a new service that acts as a proxy between Cloudflare Workers and existing databases. Hyperdrive maintains regional connection pools to reduce connection latency and caches popular read queries to improve performance. It supports PostgreSQL and PostgreSQL-compatible databases, with plans for MySQL support. The integration leverages the new Workers TCP socket API and requires minimal code changes by swapping out the database connection string.
  31. D1: open beta is hereD1 is now in open beta with increased per-database storage limits (up to 2GB) and the ability to create 50,000 databases per account. Time Travel, which allows rolling back databases to any minute in the last 30 days, is now enabled by default for all D1 databases and is free. Pricing has shifted to a row-based model, charging for reads, writes, and storage, with significant free tiers included. D1 is also being positioned for 'Workers for Platforms' to enable database-per-user deployments.
  32. D1: We turned it up to 11This post announces a major update to Cloudflare's D1 database, introducing a new storage backend that significantly boosts read (up to 20x) and write (6.8-11x) performance. It also introduces Time Travel for point-in-time recovery up to 30 days, a new console interface for querying, formal JSON function support, and Location Hints for database placement. The pricing model is detailed, including an always-free tier and integrated global read replication. The post highlights the migration of the Northwind Traders demo to the new backend and benchmarks D1 against a serverless Postgres provider.
  33. Announcing database integrations: a few clicks to connect to Neon, PlanetScale and Supabase on WorkersIntroduced 'Database Integrations' for Cloudflare Workers, enabling seamless connection to third-party databases (Neon, PlanetScale, Supabase) via HTTP. This feature simplifies discovery and authentication through an OAuth2 flow, storing credentials as encrypted environment variables. It also announces upcoming support for more databases and an integrations platform for third-party providers, along with Wrangler CLI integration and account-level environment variables.
  34. Cloudflare R2 and MosaicML enable training LLMs on any compute, anywhere in the world, with zero switching costsThis post details the integration of Cloudflare R2 object storage with MosaicML's StreamingDataset and Composer libraries to enable efficient training of large language models (LLMs). It demonstrates how to stream training data and save/load model checkpoints directly to/from R2, leveraging R2's zero-egress pricing to facilitate multi-cloud GPU utilization. Code examples are provided for writing data to R2 using `awscli` and reading data with `StreamingDataset`, as well as saving checkpoints with `Composer`. The post highlights the flexibility and cost savings achieved by decoupling data storage from compute.
  35. Use Snowflake with R2 to extend your global data lakeThis post announces a partnership between Cloudflare R2 and Snowflake, enabling users to query data stored in R2 data lakes using Snowflake and load data from R2 into Snowflake. It details the process of generating R2 API tokens and creating external stages in Snowflake using S3-compatible endpoints. It also provides SQL examples for loading data with `COPY INTO` and querying external tables.
  36. UPDATE Supercloud SET status = 'open alpha' WHERE product = 'D1';This post announces the open alpha of D1, a global relational database built on Workers. It details the included features such as dashboard management, Wrangler CLI integration for SQL file execution and migrations, and access from Workers and Pages Functions. It also highlights community-contributed tooling like d1-orm, workers-qb, d1-console, and a Kysely adapter. Limitations in the alpha include single database instances per region, concurrency queuing under high load, and temporary blocking during backups.
  37. Making static sites dynamic with Cloudflare D1This post details the initial experience of using Cloudflare D1, a new SQL database for Workers, to add dynamic comments to a static blog site. It covers setting up a new D1 database using `wrangler d1 create`, configuring bindings in `wrangler.toml`, executing SQL commands and schema files with `wrangler d1 execute`, and interacting with the database from a Hono-based Workers API for retrieving and creating comments. It highlights D1's integration with wrangler for local development and its ability to augment static sites with dynamic, relational data.
  38. Easy Postgres integration on Cloudflare Workers with Neon.techThis post introduces Neon.tech's serverless Postgres driver (`@neondatabase/serverless`) for Cloudflare Workers. It details how to set up a connection using a WebSocket proxy and a client library that shims Node.js features for Workers. The driver uses WolfSSL compiled to WebAssembly for end-to-end TLS encryption. It also demonstrates using PostGIS for geospatial queries within Workers.
  39. D1: our quest to simplify databasesIntroduced the concept of D1 Procedures, a mechanism to execute JavaScript code, including transactions, directly within the database. This addresses the challenge of mixing SQL and JavaScript code execution across different locations (Worker near user, D1 database globally distributed) and the limitations of native SQL transactions in a distributed environment. The proposed API involves defining methods in a separate JS file, accessible via `env.DB.Procedures`, and introduces a `db.transaction()` API for atomic operations within these procedures.
  40. R2 is now Generally AvailableThis post announces the General Availability of R2, Cloudflare's object storage service. It highlights key features including S3 compatibility with code examples, automatic region selection, deep integration with Cloudflare Workers (with a code example for a REST API), presigned URLs (with a code example), public buckets, and a transparent pricing model. It also outlines future plans for object lifecycles, jurisdictional restrictions, and live migration without downtime. The post also mentions that Cloudflare Images is now powered by R2.
  41. Performance isolation in a multi-tenant database environmentThis post details the implementation of gateway-level concurrency throttling in a forked version of PgBouncer to address performance isolation issues in multi-tenant Postgres clusters. It introduces the ability to dynamically set and kill existing connections per user or user/database pair, moving beyond static limits. The post also outlines future solutions including adaptive throttling inspired by TCP Vegas and tenant resource quotas.
  42. Open sourcing our fork of PgBouncerCloudflare has open-sourced its fork of PgBouncer, a PostgreSQL connection pooler. This fork includes bug fixes for authentication rejection issues when using HBA authentication and new features for runtime enforcement of per-user and per-pool connection limits. These changes enable stricter tenant performance isolation and address issues experienced by the wider open-source community.
  43. Announcing D1: our first SQL databaseThis post announces the beta of D1, Cloudflare's first SQL database built on SQLite. It highlights the developer experience, including easy database creation, Wrangler 2 integration, and simple Worker binding for access. Key features previewed include read replication, batching for multiple SQL statements in a single HTTP round-trip, embedded compute for running Worker code alongside the database, automatic backups to R2, and import/export capabilities. D1 is positioned as a serverless database for building APIs, applications, and internal tools.
  44. Introducing Relational Database ConnectorsThis post introduces the initial support for connecting Cloudflare Workers to relational databases (Postgres, MySQL) from existing infrastructure. It details the challenges of serverless database connections (TCP requirements, Node.js compatibility, connection pooling) and outlines an approach using cloudflared for secure tunnels, a shim layer to adapt socket APIs to WebSockets, and community drivers (e.g., from Deno). It also highlights future plans for native TCP support in the Workers runtime, managed connection pooling services, and global data access.
  45. Durable Objects — now Generally AvailableThis post announces the general availability and production readiness of Durable Objects. It details their core functionality: unique, single-threaded instances processing messages in-order with strongly consistent key-value storage. It highlights their benefits for global scale, concurrency, and simplifying complex applications by eliminating the need for traditional databases and their associated management. The post also outlines improvements made since early access, including enhanced reliability, performance, powering other Cloudflare products (R2, Waiting Room), customer adoption (gaming backends, status pages, collaborative whiteboards), operational improvements for high request volumes, Jurisdictional Restrictions, caching for storage operations, and simplified deployment/debugging tools. Future plans include replication for multi-threaded access, guaranteed callbacks to wake objects, and better geo-distribution.
  46. Announcing Cloudflare R2 Storage: Rapid and Reliable Object Storage, minus the egress feesIntroduced Cloudflare R2 Storage, a new object storage service with S3 API compatibility and zero egress fees. R2 is priced at $0.015 per GB/month for storage and offers a cost-effective per-operation charge above a certain threshold. It features automatic intelligent tiering for performance and cost optimization. A migration service is provided to facilitate data transfer from S3-compatible storage. R2 guarantees eleven 9s of annual durability and is designed for high availability. It integrates with Cloudflare Workers for dynamic data transformation and manipulation.
  47. Modernizing a familiar approach to REST APIs, with PostgreSQL and Cloudflare WorkersThis post introduces a tutorial for connecting to Postgres databases within Cloudflare Workers functions using PostgREST. It demonstrates how to build a REST API on the edge that communicates directly with a Postgres database. It also highlights the use of Cloudflare Tunnel for securely exposing the PostgREST endpoint and the `postgrest-js` library for interacting with PostgREST from Workers. The post discusses scaling strategies for Postgres applications by leveraging Workers' architecture for caching and performance optimization.
  48. Durable Objects: Easy, Fast, Correct — Choose threeThis post details significant improvements to the Durable Objects runtime that make them correct by default and faster. Previously, applications using Durable Objects' key/value storage API could suffer from subtle race conditions due to the interleaving of asynchronous operations (awaits) and slow disk writes. The new runtime processes writes asynchronously, ensuring data durability without blocking application logic, thus eliminating race conditions and improving performance by avoiding the need for explicit transactions or in-memory caching for correctness.
  49. AWS’s Egregious EgressThis post analyzes and criticizes AWS's egress bandwidth pricing model, highlighting its significant markups compared to wholesale bandwidth costs and the industry trend of waiving or discounting egress fees through initiatives like the Bandwidth Alliance. It contrasts AWS's 'stocks' pricing model with the industry's 'flows' model and argues that AWS's pricing strategy locks customers into their cloud. The post also touches upon the asymmetry of AWS charging for egress but not ingress, which is contrary to how wholesale bandwidth is priced. This contributes to the understanding of Cloudflare's value proposition in offering cost-effective and transparent bandwidth solutions.
  50. Empowering customers with the Bandwidth AllianceThis post introduces the Bandwidth Alliance as a mechanism to reduce egress fees for data transfer between Cloudflare and its partners. It explains how Cloudflare's global network and private interconnects enable low-cost data transfer to partner data centers. The post highlights customer benefits, including cost savings and increased choice, and mentions specific case studies and the integration with Microsoft Azure for discounted egress.
  51. Cloudflare customers can now use Microsoft Azure Data Transfer Routing Preference to enjoy lower data transfer costsThis post details the integration of Cloudflare's network with Microsoft Azure's Data Transfer Routing Preference service. It explains how Cloudflare's extensive peering and private interconnections with Azure enable customers to route traffic between Cloudflare's edge and Azure origins with reduced costs and improved performance. The post outlines the technical steps for customers to enable this feature in their Azure and Cloudflare dashboards, highlighting the benefits of direct, private interconnections for egress traffic.
  52. Announcing Cloudflare’s Database PartnersThis post announces partnerships with Macrometa and Fauna to integrate their edge-first database solutions with Cloudflare Workers. It details Macrometa's globally replicated NoSQL data platform and Fauna's globally distributed transactional database API. It also presents case studies for a bookstore demo app using Macrometa and MeetKai using Fauna and Workers, highlighting performance improvements and developer experience benefits.
  53. Durable Objects, now in Open BetaThis post announces the open beta of Durable Objects, a new paradigm for stateful serverless computing. Durable Objects provide coordination across multiple Workers and strongly consistent edge storage. Each Durable Object is defined by a JavaScript class and a unique ID, ensuring only one instance runs globally. This allows for coordination across multiple requests and enables Workers to make requests to downstream APIs and store data with a strongly consistent storage API. The post details pricing, comparing it favorably to AWS Lambda and DynamoDB, and highlights features like in-memory caching, multi-region replication by default, and strong consistency at no additional charge. It also mentions recent feature additions like smarter initial placement for user-named Objects, jurisdictional restrictions, and Wrangler support.
  54. Supporting Jurisdictional Restrictions for Durable ObjectsIntroduced Jurisdictional Restrictions for Durable Objects, allowing developers to specify a geographical region (e.g., 'eu') for data storage and processing when generating Durable Object IDs. This ensures compliance with data localization regulations without requiring developers to manage separate infrastructure. The implementation encodes jurisdictional metadata within the Durable Object ID, mapping it to specific Cloudflare data centers within the designated region. This feature is currently not supported for user-generated IDs.
  55. Workers Durable Objects Beta: A New Approach to Stateful ServerlessIntroduced Durable Objects, a new serverless primitive for stateful applications on Cloudflare Workers. Durable Objects provide globally unique, durable, and strongly consistent storage co-located with the object instance. They enable real-time coordination between clients via WebSockets and other mechanisms, allowing for the development of fully edge-hosted applications without a traditional origin server. The post details the concepts of 'Objects', 'Unique', and 'Durable', and provides examples for an atomic counter and a chat application.
  56. UtahFS: Encrypted File StorageIntroduces UtahFS, a proof-of-concept encrypted file storage system. UtahFS utilizes object storage for the underlying storage layer, breaks files into 32KB blocks connected by a skip list for efficient partial file access, and employs AES-GCM for confidentiality and Merkle Trees for integrity. It also incorporates Path ORAM to hide access patterns from the storage provider. The system is open-source and documented but not used in production at Cloudflare.
  57. Introducing Quicksilver: Configuration Distribution at Internet ScaleThis post details the replacement of Kyoto Tycoon (KT) with a new system called Quicksilver for configuration distribution at Cloudflare's internet scale. It highlights the limitations of KT, including exclusive write lock issues that degraded read latency and data corruption problems due to disabled fsync for performance. The post describes the architecture of the previous KT-based system and the new Quicksilver system, emphasizing the need for a fast, reliable, and scalable solution to distribute configuration changes to 200 cities within seconds. The technical challenges encountered with KT, such as poor concurrent access performance and operational difficulties in managing thousands of instances, are thoroughly explained. The post serves as a case study in outgrowing a datastore and building a replacement from scratch to meet evolving infrastructure demands.
  58. What’s new with Workers KV?Introduced new APIs for Workers KV: bulk upload/delete of key/value pairs, and listing keys with pagination (cursor) and prefix filtering. Increased the maximum value size limit from 2MB to 10MB and the number of namespaces per account from 20 to 100. Added a KV browser to the dashboard for managing keys and values. Enhanced Wrangler CLI integration for KV namespace creation and key management.
  59. Workers KV — Cloudflare's distributed databaseThis post announces the general availability of Workers KV, a highly distributed, eventually consistent key-value store for Cloudflare Workers. It details the CAP theorem trade-offs (Availability and Partition Tolerance over Consistency), explaining eventual consistency with an analogy. New features introduced include bulk write endpoints for efficient data import and expiring keys via `expirationTtl` and `expiration` options for time-limited data storage. The post provides code examples for using these features, such as handling billions of redirects, uploading translations via Crowdin, and blocking IPs for a limited time.
  60. Bandwidth Alliance Partners - Exciting ChoicesThis post introduces the Bandwidth Alliance, a program offering zero-egress rates for data transfer between Cloudflare and its partners. It details a solution design methodology for selecting storage providers based on customer needs (zero-egress, cost, read/write rates, volume, object size, S3 API compatibility, security) and categorizes providers based on their fit. It highlights the importance of aligning provider economics with specific use cases, particularly regarding read request pricing.
  61. Building With Workers KV, a Fast Distributed Key-Value StoreIntroduced Workers KV, a highly distributed, eventually-consistent, key-value store designed for global low-latency reads. It allows storing up to a billion keys and values with ultra-low latency anywhere in the world, enabling applications to achieve performance comparable to static content delivery. Use cases include API gateways, dynamic data serving, configuration management, and inter-system integration.
  62. Scaling out PostgreSQL for CloudFlare Analytics using CitusDBThis post details the scaling of Cloudflare's PostgreSQL-based analytics infrastructure using CitusDB. It describes the log processing pipeline, the rationale for choosing Go and Kafka, and the specific criteria for selecting CitusDB (performance, PostgreSQL compatibility, scaling, high availability, and BI query support). The CitusDB architecture, including its primary node and worker nodes for parallel query execution, is explained. The post also highlights the benefits of PostgreSQL extensions like HyperLogLog and how CitusDB scales them across multiple servers.
  63. Kyoto Tycoon Secure ReplicationThis post details the implementation of mutually authenticated TLS tunnels for Kyoto Tycoon, a distributed key-value store used for replicating sensitive data (session ticket keys, DNS data) between Cloudflare's data centers. This enhancement protects data in transit over the public internet by ensuring both the client and server authenticate each other using TLS client authentication, preventing unauthorized access and modification. The changes, implemented using OpenSSL and an internal CA, have been contributed back to the open-source Kyoto Tycoon project.
  64. Integrating Kyoto Tycoon With PostgreSQLIntroduced a Foreign Data Wrapper (FDW) for Kyoto Tycoon (KT) that leverages PostgreSQL 9.3's writable FDW capabilities. This allows data in KT to be represented as a PostgreSQL table, enabling the use of database triggers for data synchronization between PostgreSQL and KT. This integration reduces latency and simplifies synchronization logic. It also translates PostgreSQL transactions into KT transactions, exporting ACID guarantees to KT and allowing transactional operations using SQL commands.
  65. Thoughts on the AWS outage: making the cloud more resilient to failureThis post discusses the AWS outage and its impact on services relying on IaaS. It differentiates between SaaS, PaaS, and IaaS, and highlights the challenges of data synchronization and scaling in the Data & Application layer. It then introduces the Front End layer as a candidate for distributed scaling, explaining how Cloudflare's platform acts as a scalable front-end layer that can be provisioned with a DNS change, offering acceleration and protection without application layer changes. It uses the AWS outage as a case study to illustrate the resilience of Cloudflare's front-end approach compared to IaaS.