BlogsCloudflareAsynchronous Messaging and Event-Driven Architectures

Asynchronous Messaging and Event-Driven Architectures

Asynchronous Messaging and Event-Driven Architectures

16
posts
2019–2026

Cloudflare Workflows has evolved into a production-ready, serverless durable execution engine for building long-running, multi-step applications on Workers. It implements a step-based architecture with automatic retries and state persistence, ideal for complex business processes and coordinating between systems. The new `waitForEvent` API allows Workflows to pause execution and wait for external events, such as human approvals or webhooks, enabling more sophisticated human-in-the-loop scenarios. This post introduces the development of a serverless chatbot using Cloudflare Workers and Workers KV for managing service ownership, demonstrating reduced operational overhead and development time through serverless architecture and a command pattern for handling chatbot interactions.

2026

How we built saga rollbacks for Cloudflare Workflows

6/25/2026

Introduced saga rollbacks for Cloudflare Workflows, enabling developers to define compensation logic directly within `step.do()` calls. This simplifies error handling for multi-step transactions by allowing rollback functions to be passed as an option to `step.do()`, ensuring idempotency and predictable execution order during failures. The design considerations for the API, including why fluent or builder APIs were not chosen, are also detailed.

Rearchitecting the Workflows control plane for the agentic era

4/15/2026

This post details a significant rearchitecture of the Workflows control plane (which leverages Durable Objects for durable execution) to support the agentic era. It introduces new components (SousChef and Gatekeeper) to horizontally scale the system, moving away from a single Durable Object bottleneck. This enables a massive increase in concurrent instances (from 4,500 to 50,000) and instance creation rates (from 100 to 300 per second), addressing the shift towards agent-triggered workflows operating at machine speed. It also highlights the seamless migration strategy from V1 to V2.

Durable Objects in Dynamic Workers: Give each AI-generated app its own database

4/13/2026

This post introduces Durable Object Facets, a new feature that allows dynamic loading of Durable Object code within a secure sandbox (Dynamic Workers). This enables AI-generated applications to have persistent state by providing each Dynamic Worker with its own isolated SQLite database, managed by a supervisor Durable Object. This significantly expands the use cases for Dynamic Workers beyond single-use code execution.

How we use Abstract Syntax Trees (ASTs) to turn Workflows code into visual diagrams

3/27/2026

This post details the technical implementation of generating visual diagrams for Cloudflare Workflows. It explains the challenges of visualizing dynamic, code-based workflows and the solution of using Abstract Syntax Trees (ASTs) to statically derive the graph. The post covers fetching and parsing minified JavaScript code using oxc-parser, converting ASTs to a graphical representation, and handling complex code patterns like parallel execution and function/step relationships. It also touches on the underlying dynamic execution model of Workflows.

2025

How Workers powers our internal maintenance scheduling pipeline

12/22/2025

This post introduces a new maintenance scheduler built on Cloudflare Workers to automate and de-risk critical maintenance operations. It details the challenges of manual coordination, the need for a centralized 'brain', and the implementation using a graph processing approach inspired by Facebook's TAO. The post also describes the development of a fetch pipeline to address subrequest limits and improve caching efficiency, achieving a ~99% cache hit rate.

How we simplified NCMEC reporting with Cloudflare Workflows

4/11/2025

This post details the successful migration of the NCMEC reporting system from a complex, queue-based architecture to Cloudflare Workflows. It highlights the challenges of manual state management, unreliable retries, and difficult debugging in the previous system. The new implementation leverages Workflows to define a clear sequence of steps, including report creation, file gathering, uploading, and finalization, with improved observability and retry mechanisms. The code example demonstrates the structure of a Workflow entrypoint and the use of `step.do` for discrete, observable tasks.

Cloudflare Workflows is now GA: production-ready durable execution

4/7/2025

This post announces the General Availability (GA) of Cloudflare Workflows, a serverless durable execution engine for building multi-step applications on Workers. It details the core concepts of Workflows, including its step-based architecture, automatic retries, and state persistence. A significant new feature introduced is the `waitForEvent` API, which allows Workflows to pause and wait for external events, with examples of its use in human-in-the-loop scenarios and integration with external services. Pricing details are updated to include storage costs and default state expiration.

2024

Build durable applications on Cloudflare Workers: you write the Workflows, we take care of the rest

10/24/2024

This post announces the open beta of Cloudflare Workflows, a durable execution engine for building reliable, multi-step applications. It details the core concepts of Workflows, including steps, retries, and state persistence. The post also provides a high-level architectural overview of how Workflows are built on Cloudflare Workers and Durable Objects, highlighting the use of Account Controllers and the Configuration API. It showcases how Workflows can be used for complex tasks like post-processing file uploads, generating embeddings, and interacting with vector databases.

Cloudflare acquires PartyKit to allow developers to build real-time multi-user applications

4/5/2024

Cloudflare has acquired PartyKit, an open-source platform for deploying real-time, collaborative, multiplayer applications. This acquisition aims to enhance Cloudflare's serverless computing capabilities by making them more dynamic, interactive, and stateful. PartyKit leverages Cloudflare's Durable Objects and WebSockets to simplify the development of stateful serverless functions, enabling use cases such as real-time computing, collaborative editing, and multiplayer gaming. The integration will allow PartyKit projects to be created and used within existing Workers and Pages projects, with no additional charges for commercial use beyond standard service usage.

Data Anywhere with Pipelines, Event Notifications, and Workflows

4/3/2024

This post introduces Event Notifications for R2, enabling asynchronous work triggered by data changes in R2 buckets, which can be consumed by Queues. It also previews Pipelines, a new streaming ingestion service for batching and processing data at scale before writing to R2, with support for HTTP, WebSocket, and Kafka ingestion. Finally, it details Workflows, a durable execution engine built on Cloudflare Workers, designed for reliable, multi-step applications that can resume from failures, with an example of generating text embeddings using Workers AI and storing them in Vectorize.

2023

Debug Queues from the dash: send, list, and ack messages

8/11/2023

This post introduces a new debugging workflow for Cloudflare Queues directly within the Cloudflare dashboard. Users can now send test messages, list queued messages with detailed metadata, and acknowledge (remove) messages without writing any code. This significantly enhances the developer experience by providing greater transparency into queue state and simplifying the process of testing and debugging asynchronous workflows.

Cloudflare Queues: messages at your speed with consumer concurrency and explicit acknowledgement

5/19/2023

This post introduces Cloudflare Queues as a new asynchronous messaging service. It highlights two key new features: Consumer Concurrency, which automatically scales the number of consumers to match message production rates, and Explicit Acknowledgment, allowing individual message handling for more precise retry logic. The post also notes a quadrupling of message throughput and outlines future plans for R2 integration and an HTTP pull API.

2022

Announcing Pub/Sub: Programmable MQTT-based Messaging

5/12/2022

Introduced Cloudflare Pub/Sub, a programmable message bus built on the MQTT protocol, enabling real-time filtering, aggregation, and modification of messages using Cloudflare Workers. This allows for event, telemetry, or sensor data to be published from MQTT-capable clients and processed before distribution to subscribers. The platform integrates with Workers Analytics Engine for time-series data collection and can also write to R2 or KV. Future plans include broader MQTT v5.0 support, enhanced developer tooling, and support for non-MQTT clients.

Durable Objects Alarms — a wake-up call for your applications

5/11/2022

Introduced Durable Objects Alarms, a new feature that allows Durable Objects to schedule their own invocation at a future time via an `alarm()` handler. Alarms guarantee at-least-once execution and automatically retry with exponential backoff on failure. This feature is enabled via the `durable_object_alarms` compatibility flag and uses the storage layer for scheduling, ensuring fault tolerance and at-edge execution. The post provides a code example demonstrating how to use alarms for batching requests within a Durable Object.

2021

Introducing Project Fair Shot: Ensuring COVID-19 Vaccine Registration Sites Can Keep Up With Demand

1/22/2021

Introduced Project Fair Shot, offering Cloudflare's new Waiting Room service for free to organizations distributing COVID-19 vaccines. The Waiting Room service is designed to protect registration websites from crashing under load by implementing a virtual queue. This service can be deployed in front of any existing web registration site without code changes or hardware installation, requiring only a DNS change and Waiting Room configuration. The technology was moved up in launch to address the critical need for vaccine registration websites.

2019

Rapid Development of Serverless Chatbots with Cloudflare Workers and Workers KV

4/25/2019

This post details the development of a serverless chatbot using Cloudflare Workers and Workers KV to manage service ownership. It outlines the requirements for adding, deleting, querying, and exporting service ownership information. The implementation uses a Command pattern with a ServiceDirectory abstraction and Workers KV for state persistence. The post also covers the initial setup of a Google Chatbot integration and provides example code for the Worker, CommandFactory, ServiceDirectory, AddCommand, and QueryCommand, along with unit tests.