Dogfooding at scale: migrating cdnjs to Cloudflare’s Developer Platform

This post details the migration of the cdnjs publishing pipeline from a distributed system involving GCP Functions, VMs, and GitHub to an integrated Cloudflare Developer Platform architecture. Key technical contributions include: establishing R2 as the single source of truth for file content, using KV for metadata, replacing an internal caching layer with Workers Cache, and implementing a durable ingestion pipeline using Cloudflare Workflows and Queues. It also describes the use of a Rust compression service in a container and a Durable Object as a counter for managing per-package workflows. The post highlights the resolution of previous pain points such as lack of shared tracing, split-brain storage, pipeline fragility, and GitHub repository limitations.