Asynchronous Messaging and Event-Driven Architectures

18 posts · Dec 2012 – Jun 2026

What this trail covers

Cloudflare Workflows, a serverless durable execution engine for building long-running, multi-step applications on Workers, is now Generally Available. It offers production-ready reliability with step-based architecture, automatic retries, and state persistence. New capabilities include the `waitForEvent` API for human-in-the-loop interactions and AI agent integration. Pricing now includes storage costs for persisted state, with a grace period for the new storage charges. Workflows continue to scale and evolve with new features.

All Cloudflare trails

Jun 2026How we built saga rollbacks for Cloudflare WorkflowsIntroduced saga rollbacks for Cloudflare Workflows, allowing developers to define compensation logic directly within `step.do()` calls. This feature simplifies the implementation of durable transactions by providing a declarative way to handle failures and ensure state consistency. The API design prioritizes integration with existing `step.do()` primitives and leverages promise pipelining concepts for efficient execution. Rollback handlers are designed to be idempotent and execute in reverse step-start order upon workflow failure.
May 2026When "idle" isn't idle: how a Linux kernel optimization became a QUIC bugThis post details a bug discovered in Cloudflare's quiche implementation of the CUBIC congestion control algorithm. The bug caused the congestion window (cwnd) to become permanently pinned at its minimum after a congestion collapse event, preventing recovery. The issue stemmed from an optimization ported from the Linux kernel's CUBIC implementation that adjusted the epoch start time based on idle periods. In quiche's user-space implementation, this adjustment, combined with the way ACKs trigger sending, led to a rapid oscillation between recovery and congestion avoidance states, locking cwnd at 2700 bytes. The post traces the root cause back to a specific Linux kernel commit and its subsequent fix, highlighting how the user-space implementation of QUIC differed in its handling of idle periods. A one-line fix in quiche was identified and implemented to resolve this oscillation and allow for proper cwnd growth.
May 2026Introducing Dynamic Workflows: durable execution that follows the tenantThis post introduces Dynamic Workflows, a library that enables a single Worker (the Worker Loader) to route `create()` calls to different tenant's code and dispatch `run(event, step)` calls back to that same code when the workflow executes. It achieves this by wrapping the standard `WORKFLOWS` binding with tenant-specific metadata and using `createDynamicWorkflowEntrypoint` to dynamically load tenant code. The system caches tenant Worker isolates for reuse across workflow steps, allowing for per-tenant or per-request workflow logic with minimal overhead. The underlying primitive enabling this is Dynamic Workers, which handle the dynamic loading, sandboxing, and routing of tenant code.
Apr 2026Rearchitecting the Workflows control plane for the agentic eraThe Workflows control plane has been rearchitected from V1 to V2 to support increased concurrency and creation rates driven by agentic workloads. V1 relied on a single account-level Durable Object (DO) as a bottleneck. V2 introduces two new components: SousChef, which manages metadata and lifecycle for subsets of instances per workflow, and Gatekeeper, a leasing system for distributing concurrency slots across SousChefs. This distributed architecture ensures that the source of truth for instance existence is its Engine, and the Account singleton stores minimal metadata with an invariant maximum request count. The migration process involved seamlessly transitioning live traffic to the new V2 control plane without downtime.
Apr 2026Durable Objects in Dynamic Workers: Give each AI-generated app its own databaseThis post introduces Durable Object Facets, a new feature that allows dynamic loading of Durable Object code within a secure sandbox (Dynamic Workers). This enables AI-generated applications to have persistent state by providing each Dynamic Worker with its own isolated SQLite database, managed by a supervisor Durable Object. This significantly expands the use cases for Dynamic Workers beyond single-use code execution.
Advertisement
Mar 2026How we use Abstract Syntax Trees (ASTs) to turn Workflows code into visual diagramsThis post details the technical implementation of generating visual diagrams for Cloudflare Workflows. It explains the challenges of visualizing dynamic, code-based workflows and the solution of using Abstract Syntax Trees (ASTs) to statically derive the graph. The post covers fetching and parsing minified JavaScript code using oxc-parser, converting ASTs to a graphical representation, and handling complex code patterns like parallel execution and function/step relationships. It also touches on the underlying dynamic execution model of Workflows.
Dec 2025How Workers powers our internal maintenance scheduling pipelineThis post introduces a new maintenance scheduler built on Cloudflare Workers to automate and de-risk critical maintenance operations. It details the challenges of manual coordination, the need for a centralized 'brain', and the implementation using a graph processing approach inspired by Facebook's TAO. The post also describes the development of a fetch pipeline to address subrequest limits and improve caching efficiency, achieving a ~99% cache hit rate.
Nov 2025A closer look at Python Workflows, now in betaThis post introduces Python support for Cloudflare Workflows, enabling developers to orchestrate multi-step applications using Python. It details the integration of Python into the Workers runtime via Pyodide, the use of FFI for JavaScript interop, and the creation of a Pythonic SDK. Key contributions include the use of decorators for idiomatic Python workflow definition (e.g., `@step.do`), snake_case conversion for method names, and support for declarative DAG workflows with `asyncio.gather` and custom decorators for managing dependencies and concurrency. The post also explains the underlying RPC mechanisms and language bridging required for Python Workers.
Apr 2025How we simplified NCMEC reporting with Cloudflare WorkflowsThis post details the migration of Cloudflare's NCMEC reporting system from a queue-based architecture using Workers Queues to Cloudflare Workflows. The new system simplifies the process by defining explicit workflow steps for report creation, file gathering, uploading, and finalization. Key improvements include enhanced observability via the Workflows dashboard, simplified debugging by tracing a single workflow, more reliable error handling with automatic retries and exponential backoff, and reduced operational overhead by eliminating the need for manual intervention and a 'Reaper' job. The code example demonstrates the use of `WorkflowEntrypoint`, `WorkflowEvent`, and `WorkflowStep` to orchestrate these tasks.
Apr 2025Cloudflare Workflows is now GA: production-ready durable executionThis post announces the General Availability (GA) of Cloudflare Workflows, a serverless durable execution engine. It details the core functionality of Workflows, emphasizing its step-based architecture, automatic retries, and state persistence for building resilient, multi-step applications. A key new feature highlighted is the `waitForEvent` API, which allows workflows to pause and wait for external events (human approval, webhooks, etc.) before continuing execution. The post also discusses integration with AI agents and the new pricing model, which now includes storage costs for persisted workflow state. The example code demonstrates defining a workflow with multiple steps, including custom retry logic for payment processing and the use of `waitForEvent` for human approval in a code review scenario.
Oct 2024Build durable applications on Cloudflare Workers: you write the Workflows, we take care of the restThis post announces the open beta of Cloudflare Workflows, a durable execution engine for building reliable, multi-step applications. It details the core concepts of Workflows, including steps, retry strategies, and state persistence. The post also provides a high-level architectural overview of how Workflows is built on Cloudflare Workers and Durable Objects, highlighting the use of stateless Workers for configuration APIs and binding shims, and Durable Objects for account controllers and workflow engines. It introduces the concept of a managed platform that handles compute and storage infrastructure, allowing developers to focus on application logic. The post also showcases example code for defining workflows and steps, and mentions the integration with other Cloudflare services like R2 and Workers AI.
Apr 2024Cloudflare acquires PartyKit to allow developers to build real-time multi-user applicationsCloudflare has acquired PartyKit, an open-source platform for deploying real-time, collaborative, multiplayer applications. This acquisition aims to enhance Cloudflare's serverless computing capabilities by making them more dynamic, interactive, and stateful. PartyKit leverages Cloudflare's Durable Objects and WebSockets to simplify the development of stateful serverless functions, enabling use cases such as real-time computing, collaborative editing, and multiplayer gaming. The integration will allow PartyKit projects to be created and used within existing Workers and Pages projects, with no additional charges for commercial use beyond standard service usage.
Apr 2024Data Anywhere with Pipelines, Event Notifications, and WorkflowsThis post introduces Event Notifications for R2, enabling asynchronous work triggered by data changes in R2 buckets, which can be consumed by Queues. It also previews Pipelines, a new streaming ingestion service for batching and processing data at scale before writing to R2, with support for HTTP, WebSocket, and Kafka ingestion. Finally, it details Workflows, a durable execution engine built on Cloudflare Workers, designed for reliable, multi-step applications that can resume from failures, with an example of generating text embeddings using Workers AI and storing them in Vectorize.
Aug 2023Debug Queues from the dash: send, list, and ack messagesThis post introduces a new debugging workflow for Cloudflare Queues directly within the Cloudflare dashboard. Users can now send test messages, list queued messages with detailed metadata, and acknowledge (remove) messages without writing any code. This significantly enhances the developer experience by providing greater transparency into queue state and simplifying the process of testing and debugging asynchronous workflows.
May 2023Cloudflare Queues: messages at your speed with consumer concurrency and explicit acknowledgementThis post introduces Cloudflare Queues as a new asynchronous messaging service. It highlights two key new features: Consumer Concurrency, which automatically scales the number of consumers to match message production rates, and Explicit Acknowledgment, allowing individual message handling for more precise retry logic. The post also notes a quadrupling of message throughput and outlines future plans for R2 integration and an HTTP pull API.
Mar 2023Accelerate building resiliency into systems with Cloudflare WorkersThis post details the implementation of a weighted traffic routing module for email service providers using Cloudflare Workers. The module allows for dynamic routing of email traffic between multiple vendors based on configurable weights, enabling rapid failover during outages. The implementation uses a simple ratio-based random picker and environment variables for configuration, deployed globally within milliseconds via Cloudflare Workers. This approach minimizes code changes in existing services and offloads scaling and load balancing to the Workers platform.
Mar 2023How we built an open-source SEO tool using Workers, D1, and QueuesThis post introduces Prospector, an open-source SEO tool built using Cloudflare Workers, D1, and Queues. It demonstrates how to combine these services for a functional application, including data modeling for notifiers and URLs, sitemap parsing, and using Queues for scheduled scraping and notifications. The use of Hono framework for API routes and improved TypeScript bindings for Workers are highlighted.
May 2022Announcing Pub/Sub: Programmable MQTT-based MessagingIntroduces Cloudflare Pub/Sub, a new programmable message bus service based on the MQTT protocol. It enables developers to publish event, telemetry, or sensor data from MQTT-capable clients. Cloudflare Workers can be used to filter, aggregate, and modify messages in real-time as they are published, before distribution to subscribers. The service acts as a programmable event router or a hook into persistent storage like R2 or KV. An example demonstrates using Workers to process transaction data, write metrics to Workers Analytics Engine, and query them with SQL. Future plans include full MQTT v5.0 support, Wrangler CLI integration, native Worker bindings, and support for HTTP/WebSocket clients.
May 2022Durable Objects Alarms — a wake-up call for your applicationsIntroduced Durable Objects Alarms, a new feature that allows Durable Objects to schedule their own invocation at a future time via an `alarm()` handler. Alarms guarantee at-least-once execution and automatically retry with exponential backoff on failure. This feature is enabled via the `durable_object_alarms` compatibility flag and uses the storage layer for scheduling, ensuring fault tolerance and at-edge execution. The post provides a code example demonstrating how to use alarms for batching requests within a Durable Object.
Jan 2021Introducing Project Fair Shot: Ensuring COVID-19 Vaccine Registration Sites Can Keep Up With DemandIntroduced Project Fair Shot, offering Cloudflare's new Waiting Room service for free to organizations distributing COVID-19 vaccines. The Waiting Room service is designed to protect registration websites from crashing under load by implementing a virtual queue. This service can be deployed in front of any existing web registration site without code changes or hardware installation, requiring only a DNS change and Waiting Room configuration. The technology was moved up in launch to address the critical need for vaccine registration websites.
Apr 2019Rapid Development of Serverless Chatbots with Cloudflare Workers and Workers KVThis post details the development of a serverless chatbot using Cloudflare Workers and Workers KV to manage service ownership. It outlines the requirements for adding, deleting, querying, and exporting service ownership information. The implementation uses a Command pattern with a ServiceDirectory abstraction and Workers KV for state persistence. The post also covers the initial setup of a Google Chatbot integration and provides example code for the Worker, CommandFactory, ServiceDirectory, AddCommand, and QueryCommand, along with unit tests.
Mar 2018Squeezing the firehose: getting the most from Kafka compressionThis post details the performance benefits of Kafka compression, specifically exploring the trade-offs between CPU usage and disk/network I/O for various compression codecs (gzip, snappy, lz4) on high-throughput data streams like HTTP logs. It highlights the evolution of Kafka's log format to support batch compression, significantly improving efficiency. The post also details performance testing with the sarama Go client library and analyzes bottlenecks encountered at Cloudflare, including disk contention and network saturation, which compression aims to alleviate.
Dec 2012Pushing Nginx to its limit with LuaThis post introduces the use of Lua scripting within Nginx to augment its capabilities, specifically for log aggregation. It highlights the benefits of Nginx+Lua, including its asynchronous nature, speed (leveraging LuaJIT), and ease of integration. The post provides installation instructions via ngx_openresty and demonstrates a practical example of building a log aggregator. It also discusses limitations and the availability of libraries for various protocols (SQL, memcached, Redis, DNS). The author emphasizes Lua's expressiveness and approachability for adding new features to Nginx, comparing it to custom C modules.