Artificial Intelligence Integration & Impact

Oct 2026Introducing Clef-omni with full multimodality, plus a faster Clef and a cheaper Clef-flashIntroduces Clef-omni, a multimodal decision model capable of processing audio, video, image, and text input, built on a Qwen3-Omni-30B-A3B-Instruct MoE foundation. It skips output token generation, focusing on scoring modalities directly via a two-stage attention routing mechanism and a built-in lexical grammar for schema-constrained scoring. Clef-flash pricing has been reduced to $0.038 per M input tokens with a context window of 24k, while Clef remains at $0.24 per M input tokens with a 64k context window. Clef model performance has been improved by 1.7x-2.0x median speedup through serving infrastructure optimizations, including the adoption of SGLang for serving. Benchmarks demonstrate competitive performance for Clef-omni, Clef, and Clef-flash across various tasks and modalities.

Oct 2026Introducing Web Search API via AI GatewayThis post introduces the integration of a Web Search API into Cloudflare's AI Gateway. This enables AI agents to fetch live internet data, grounding their responses with current information. The feature supports multiple web search providers (Ceramic.ai, Exa, Linkup) and can be accessed via REST API and Workers bindings. It draws from AI Gateway credits for usage and provides observability logs. Partners are committed to Cloudflare's bot crawling standards. Future work includes native server tools within AI Gateway.

Oct 2026Introducing Clef: our open-source decision models, and new RL fine-tuning platformIntroduces Clef and Clef-flash, new Cloudflare-trained decision models hosted on Workers AI. These models are Jev-API compatible, open-sourced on Hugging Face, and offer multimodal classification (image and text) with a 64k context window. Benchmarks show competitive accuracy and significantly lower latency compared to existing models like Jev. The training approach leverages Qwen as a base model and employs a specialized two-stage attention routing process for non-autoregressive inference. Also announces a new RL product for fine-tuning these models.

Oct 2026One year later: Sovereign AI and the fight for choiceThis post announces the addition of two new European open-source AI models, EuroLLM and Apertus, to Cloudflare's Workers AI platform. EuroLLM supports 24 EU languages and was trained on MareNostrum 5, outperforming similar-sized models on EU multilingual benchmarks. Apertus, Switzerland's first large-scale open multilingual model, was trained on over 15 trillion tokens across 1500+ languages and emphasizes privacy compliance. The post also details the development and launch of hands-on workshops for government agencies and critical infrastructure operators to build AI defense harnesses, which coordinate multiple AI models for vulnerability hunting and threat prioritization. This harness is open-sourced and designed to work with any model, closed or open, to provide resilient AI defenses.

Oct 2026AI Search is now generally availableThis post announces the general availability of AI Search, highlighting expanded multimodal capabilities including native image embeddings (using Qwen3-VL-Embedding model) and OCR for PDFs. It details the query lifecycle, including optional query rewriting, embedding (direct for images with multimodal models, or via caption conversion for text-only models), parallel vector and keyword search, fusion, and optional reranking. The post also introduces larger file support (up to 10 MiB) and clarifies billing for AI Search, which will start on November 1, 2026, with a revised free tier allocation for semantic and full-text queries. Future work includes video and audio processing, keyword search engine refactoring, and simplified index creation for websites.

Sep 2026Cut your AI spend with AI Gateway's Auto RouterIntroduced the Auto Router feature for AI Gateway, enabling intelligent routing of AI requests to the most capable and cost-effective model. This feature uses a multi-head classification model running on Workers AI to assess task categories and request dimensions (complexity, ambiguity, stakes, context dependence). It then combines these signals with model benchmark results and token prices to calculate utility and select the optimal model, aiming to reduce AI spend by up to 30% while maintaining comparable performance to frontier models. The router also accounts for cache read/write costs and model switching penalties. The post details the two-stage architecture (classifier and scoring matrix) and its benefits for legibility and adaptability. Future plans include expanding model support, zero-data-retention, provider capacity accounting, and reasoning level selection.

Sep 2026Identify AI model overuse with User InsightsThis post introduces enhancements to Cloudflare's User Insights for AI Gateway, focusing on providing deeper context for AI usage. Key contributions include: 1. Model Overkill View: Identifies conversations where the selected model may be more capable than the task requires, enabling teams to investigate potential cost savings and model appropriateness. 2. Task Analysis: Categorizes conversations by work type (coding, research, summarization, etc.) to understand how AI is being used. 3. Turn Analysis: Tracks the number of exchanges per task to assess conversation length and potential inefficiencies. 4. Integration with Auto Router: These insights power the Auto Router (in beta) to automatically route requests to appropriate models based on task, cost, and model fit. 5. Classification Engine: Details the asynchronous Cloudflare Worker that processes AI Gateway logs to generate these signals, using Durable Objects for metadata and R2 for log bodies. The classification is out-of-path and does not add latency to user responses.

Sep 2026Pay Per Use: when AI uses your work, you should get paidIntroduces 'Pay Per Use', a new beta feature that enables publishers to monetize content used by AI products. This system allows publishers to define pricing for specific content uses and receive payments based on reported usage by AI buyers. It includes a mechanism for AI companies to identify their crawling activity using Verified Bots and report usage via a single API. Cloudflare handles enrollment, usage reporting, billing, and payment settlement between buyers and publishers.

Aug 2026Introducing Adaptive Intelligence: undermining the economics of every bot attackIntroduces Adaptive Intelligence, a new bot detection engine that shifts from deterministic rule-based systems to a non-deterministic, adaptive approach. Key components include continuous ML model retraining on live traffic, disposable rule generation that is retired before attackers can adapt, and learning from customer feedback and internal detections. The initial launch focuses on the continuously retraining ML model that aggregates network signals to measure the probability of automated abuse.

Aug 2026BotBase for Operators: A clearer path to joining Cloudflare's directory of bots and agentsThis post introduces 'BotBase for Operators,' a new interface within the Cloudflare dashboard that provides bot operators with transparency and control over their bot submissions. Key features include a dedicated submission history with status tracking (Waiting for review, Accepted, Rejected), the ability to edit existing submissions, and an updated submission form that uses a pragmatic taxonomy based on bot behavior, content use (Content Signals), and operator classification (direct vs. intermediary). The review process has been automated to expedite submissions by performing checks for duplicates, user-agent specificity, and verification methods (IP lists, reverse DNS, Web Bot Auth).

Aug 2026Say it once: introducing Bot Preference SyncIntroduces Bot Preference Sync, a feature that automatically updates a website's robots.txt file to reflect AI bot preferences (Search, Agent, Training) configured in the Cloudflare dashboard. This eliminates manual maintenance of robots.txt for AI traffic control and enhances transparency by ensuring stated preferences align with edge enforcement. It also refines the 'Disallow Training' option to allow cooperating mixed-use crawlers that meet specific transparency criteria (respecting 'no training' preferences, opt-out for AI summaries, URL-level visibility, and proof of no impact on search results) to still access content for search indexing.

Aug 2026Everything we launched during Agents WeekThis post is a summary of multiple product launches and announcements made during 'Agents Week'. It details the evolution of Cloudflare's AI platform towards an 'Agentic Internet', introducing new runtimes (@cloudflare/computer), communication primitives (Workers RPC, gRPC support), development lifecycles (ADLC), security models (Agent Access Model, Wallets), and observability tools (AI Gateway, Radar Researcher). It also covers advancements in model serving efficiency, CI/CD for agents, and web content optimization for AI discovery (AEO). The post highlights the unification of Workers AI and AI Gateway into a single control plane and the introduction of new tools for agent interaction with websites (WebMCP) and data (AI Search).

Aug 2026Unveiling good and bad behaviors on the Agentic InternetThis post introduces the concept of 'Agentic Internet' and the Web Integrity & Trust team's strategy for detecting and analyzing good and bad behaviors. It details the distinction between Risk and Trust in bot detection, highlighting the importance of trust built over time. The post discusses the evolution of BotBase to track both good and 'less-than-good' bots, emphasizing transparency. It introduces Precursor, a continuous client-side system for detecting subtly inhuman bot traffic, and shares usage statistics and learnings from its launch, including the observation that suspicious behavior often happens mid-session and that behavior can shift between human and agentic. An interactive demo, Precursor Trace, is launched to simulate cursor movement assessment. The upcoming Adaptive Intelligence engine is announced, which will continuously learn and self-adjust. Advanced mitigations like unpredictable responses and AI Labyrinth are discussed as ways to influence bot behavior.

Aug 2026Introducing Radar Researcher: An AI tool for exploring Internet data in plain languageThis post introduces Radar Researcher, a new AI-powered tool built on Cloudflare's developer platform that allows users to explore Internet data from Cloudflare Radar using natural language queries. It leverages Cloudflare Workers, Durable Objects, Workers AI (with open models like Kimi K2.7), AI Gateway, and the Cloudflare MCP server with Code Mode to interact with Radar's API. The tool translates natural language questions into API queries, generates interactive charts, and provides explanations and suggested follow-up questions. It also allows users to 'explain' existing charts with AI. This represents an evolution in how users can access and interact with complex datasets.

Aug 2026Unifying Workers AI and AI Gateway into a single AI control planeThis post details the convergence of Workers AI and AI Gateway into a single AI control plane. It introduces unified entrypoints via a Workers binding and a REST API, enabling automatic observability, logging, and cost attribution for all AI inference traffic without explicit AI Gateway setup. It also announces the ability to use AI Gateway credits for Workers AI usage, unified billing, and elevated rate limits for Workers AI models when using unified billing. Future plans for model-first routing and intelligent routing are outlined.

Aug 2026Building an open Agentic Internet: readable, discoverable, callable, and payableThis post introduces several new protocols and tools for the Agentic Internet: Web Bot Auth for bot identity, PACT (Private Access Control Tokens) for anonymous vouching, Markdown for Agents for server-side content optimization for AI, WebMCP for exposing site actions directly to agents, Kitesurf (a browser for agents running on Workers), AI Search for agent discoverability, Agent Engine Optimization (AEO) for measuring agent visibility, Wallets for agent payments, and Monetization Gateway for domain owners to set up payments. It also discusses the philosophical shift towards an 'Agentic Internet' where interactions are readable, discoverable, callable, and payable, moving away from human-centric web paradigms.

Aug 2026Cloudflare AI Search: give your agents a search engine for your dataThis post introduces Cloudflare AI Search, a new product that simplifies the process of building AI-powered search solutions. It automates the crawling, ingestion, embedding, and retrieval of data from various sources, including websites and individual files. Key features include a 'Discover' option for websites without sitemaps, a single public endpoint for searching across multiple instances, custom domain support, and an EmDash CMS plugin. The post also details how AI Search is used internally to power the Cloudflare Dev Stack MCP Server, Cloudflare.com, and Developer Docs. It provides a preview of the pricing model, highlighting free embedding and reranking with select Workers AI models.

Aug 2026From ranking to recommended: get your site ready to thrive in the age of AI agentsIntroduces Agent Readiness and Answer Engine Optimization (AEO) tools. Agent Readiness provides diagnostics on site accessibility and content discoverability for AI agents, including checks for robots.txt, sitemaps, Markdown versions, and API catalogs. AEO infers industry and category from a site to probe AI assistants (Claude, GPT) with prompts to measure Citation Rate, Prominence, Mention Rate, and Share of Voice. It also tracks AI operator activity on sites, including crawl traffic and errors encountered.

Aug 2026How we’re rethinking work at Cloudflare with Cloudflare OSThis post details the internal development and launch of Cloudflare OS, a platform designed to enable safe and productive use of AI and agents within Cloudflare. It outlines the principles guiding AI adoption, the iterative process of identifying user needs through a 'magic AI email bot,' and the technical implementation of Cloudflare OS. Key components include the Engineering Codex for AI-assisted code review and design, and a web-based harness for running automated workflows and skills. The post emphasizes the integration of existing Cloudflare products (Workers, Access) with custom services to ensure security and control over AI-driven operations.

Aug 2026Catching rogue AI behavior with identity-aware analyticsIntroduces identity-aware AI Gateway with Cloudflare Access integration, allowing authentication via SAML providers and policy setting for access control. Adds `cf.user_id` to request metadata for per-user logging and spend tracking. Introduces User Insights tab for behavioral baselining and anomaly detection, comparing session costs against a user's p95 and an account's p99 to identify deviations. Implements a dollar floor to filter out low-cost anomalies. Outlines future work on task-based smart routing and prompt classification.

Aug 2026Cloudflare OS: an open platform for agents, apps, and workThis post introduces Cloudflare OS, an open platform for agents, apps, and work, built on a new foundation for security and governance. It enables organizations to deploy their own AI agents and workspaces, grounded in company-specific context and skills. Key components include an agent workspace with isolated runtime, a new security and governance framework with Gatekeepers for controlled access to internal systems, and a platform for building and sharing personal, modifiable apps. The platform records agent observations to enforce policies and prevent data exfiltration. It integrates with existing Model Context Protocol (MCP) servers and provides a typed binding system for resource access.

Aug 2026WriteGuard: Fine-grained controls for MCP ServersIntroduced WriteGuard, a system for fine-grained control over AI agent actions on MCP servers. WriteGuard implements a shared policy, attribution, and auditing layer that sits between MCP clients and servers. It allows for defining risk tiers for tools, enabling/disabling them, and configuring labeling for agent attribution. WriteGuard also generates scrubbed audit events for agent activity, providing a centralized view of machine-speed actions across MCP-enabled systems.

Aug 2026Announcing Cloudflare Wallets: The programmable wallet for the agentic InternetIntroduces Cloudflare Wallets, a new system for programmable payments for AI services and agentic commerce. This includes Account Wallets for human users and Virtual Wallets for AI agents, with features for setting spending limits, allowances, and transaction caps. It also proposes human-readable identifiers for agents, building on existing bot management and identity primitives, and integrates with the Monetization Gateway and the x402 protocol for micropayments.

Aug 2026How Cloudflare enforces engineering standards using AIThis post introduces the Cloudflare Codex, a system for organizing and enforcing engineering standards. It details the Codex's RFC-based structure, governance model, and lifecycle states (approved vs. enforced). It also describes how Codex statements are extracted and compacted for consumption by AI agents. The post highlights three key consumers of the Codex: the AI code reviewer (which flags violations and blocks merges), the spec reviewer (which evaluates technical designs before implementation), and the incident report reviewer (which assesses postmortems for completeness and quality. It also mentions the development of language-specific linters and a CLI for local AI code reviewer execution.

Aug 2026How we built a software factory to drive Astro’s GitHub issue count to zeroThis post details the development and implementation of an automated issue triage pipeline for the Astro open-source project, leveraging AI agents. It describes the creation of the Flue framework, an open, platform-agnostic system for building durable agents and workflows, and the triagebot-action, a GitHub Action that automates the process of reproducing, diagnosing, and fixing bugs. The contribution highlights the use of isolated AI subagents, state machines driven by issue labels, and the integration with GitHub Actions for transparency and auditability. It also discusses the benefits of this automation, including reducing maintainer burnout and improving code quality by identifying architectural or documentation issues.

Aug 2026Introducing: Cloudflare AgentsIntroduces Cloudflare Agents, a platform for deploying and managing hosted agents on Cloudflare, starting with observability. Launches agent tracing, which provides agent-aware telemetry for model calls, tool execution, and token usage, integrating with OpenTelemetry-compatible agent harnesses (Think, Flue, AI SDK). Adds a dedicated Agents view in the Cloudflare dashboard for visualizing and debugging agent behavior through session replays and execution traces. Details how to enable agent tracing via wrangler.jsonc and specific setup instructions for different agent stacks. Mentions future support for direct OpenTelemetry API integration within Workers and export of traces to OTLP providers. Outlines pricing for Workers tracing, which underpins agent tracing, with a beta period offering free tracing.

Aug 2026Smaller, faster, safer: running Kimi and GLM at scaleThis post details three techniques for optimizing the serving of large, long-context, mixture-of-experts AI models (Kimi K-series and GLM) on Cloudflare's Workers AI platform: 1. KV cache quantization from BF16 to FP8, which doubles the context length and increases concurrent requests by 41% at a lower cost per token, with no measurable accuracy degradation. 2. Model weight compression from FP8 to INT4, reducing checkpoint size by 40% and per-GPU memory, leading to a 55% speedup in decode latency at low concurrency, with model accuracy within 0.8 points. 3. KV cache integrity checking, a defense layer that adds a tag to cache pages and checks mappings before decode operations, incurring less than 1% overhead on throughput and tail latency. The post also highlights the use of SGLang for inference serving and the collaboration with its team.

Jul 2026Announcing the Monetization Gateway: charge for any resource behind Cloudflare via x402This post introduces the Cloudflare Monetization Gateway, a new capability that allows customers to charge for any resource protected by Cloudflare (web pages, datasets, APIs, etc.) using the x402 protocol and stablecoins. It directly addresses the shift in web business models driven by AI agents, proposing usage-based pricing and micropayments as the new standard. The gateway handles payment verification and enforcement at the edge, simplifying implementation for businesses and enabling new revenue opportunities for creators and API providers.

Jul 2026Content Independence Day, one year on- building the business model for the agentic InternetThis post details the significant shift in internet traffic composition driven by AI adoption, with AI training crawlers now comprising 52% of identified crawler requests. It highlights the decline of traditional referral traffic and the rise of the 'agentic Internet' where over 50% of traffic is non-human. Cloudflare's 'Content Independence Day' initiative is presented as a catalyst for a new market where content owners can control and monetize their data for AI training. The post emphasizes the emergence of publisher-AI licensing agreements and the challenges posed by mixed-use crawlers, particularly from Google, which blur the lines between discovery and AI training access. Cloudflare's role at the intersection of this emerging economy is underscored, providing tools for transparency, control, and scarcity to facilitate a compensated content ecosystem.

Jul 2026Making AI search smarterThis post introduces initiatives to make AI search smarter by leveraging network signals for content freshness and relevance, aiming to reduce unnecessary crawling and establish new economic models for content creators. It also details the evolution of 'Pay Per Crawl' to 'Pay Per Use' with partners like Ceramic.ai and You.com, enabling content owners to be compensated based on usage in AI answers and providing new reporting for answer engine optimization.

Jul 2026Your site, your rules: new AI traffic options for all customersThis post introduces a significant evolution in Cloudflare's AI integration by providing granular controls for managing AI bot traffic. It moves beyond a simple 'block AI' approach to a pragmatic taxonomy based on bot behavior (Search, Agent, Training) and content usage. New default settings will be implemented, and BotBase, a comprehensive bot database, is launched for Enterprise customers to enhance visibility and control over automated traffic.

Jul 2026Unmasking the crawls with Attribution Business InsightsIntroduces the Attribution Business Insights dashboard for Cloudflare Bot Management customers. This dashboard provides granular data on bot traffic, specifically differentiating between human, non-AI bot, and AI bot traffic. It offers site-wide and per-bot-operator crawl-to-referral ratios, and classifies AI crawlers into 'Training', 'Search', and 'Agent' categories based on their behavior. The goal is to empower website owners with data to make informed decisions about AI traffic and content access.

Jun 2026Build your own vulnerability harnessThis post details the development of a model-agnostic 'vulnerability harness' for AI-driven security analysis. It describes the evolution from a single-repo 'skill' to a fleet-wide scanning pipeline, emphasizing state management, persistence, and cross-repo reasoning. The harness is divided into a Vulnerability Discovery Harness (VDH) and a Vulnerability Validation System (VVS), using different models for each stage to ensure adversarial validation and model interchangeability. Key components include Recon, Hunt, Validate, Gapfill, Dedup, Trace, and Feedback agents, all orchestrated to manage context exhaustion and ensure robust vulnerability discovery across diverse codebases.

Jun 2026Bringing more agent harnesses and frameworks to Cloudflare, starting with FlueThis post introduces the Cloudflare Agents SDK as a foundational layer for building production-grade AI agents, providing primitives like durable execution, dynamic code execution, and a durable filesystem. It also highlights Flue, a new open-source framework built on the SDK, which simplifies agent development through a declarative approach and integrates with various platforms. The post details how the Agents SDK addresses key challenges for agent harnesses, such as durable execution via Fibers, secure code execution using Code Mode with Dynamic Workers, and persistent file systems with @cloudflare/shell, enabling agents to run reliably at scale.

Jun 2026Growing the Cloudflare AI team with talent from Ensemble AIThe acquisition of Ensemble AI brings expertise in model compression and efficient inference architectures, specifically NdLinear and NdLinear-LoRA, to Cloudflare. These techniques aim to reduce parameter count and compute for transformer models by operating directly on multidimensional activations. This complements existing efforts in quantization and vector quantization, with a focus on improving the economics of serving large language models and advanced AI architectures by enhancing model efficiency, GPU utilization, and scalable deployment within Cloudflare Workers AI.

Jun 2026Your AI bill is out of control. Cloudflare can fix it now.This post introduces spend controls for Cloudflare AI Gateway, allowing users to set budgets in dollars, track cumulative spend across models, providers, and custom attributes, and configure actions (blocking or routing to fallback models) when limits are reached. It also announces a closed beta for identity-driven budgets and policies, which integrate with Cloudflare Access and identity providers to enable per-user and per-team AI usage controls. This allows for verified, automatic attribution of AI spend and granular policy enforcement based on user identity and group membership.

May 2026Announcing Claude Compliance API support with Cloudflare CASBThis post announces the integration of Cloudflare CASB with Anthropic's Claude Compliance API. This integration allows security and compliance teams to monitor Claude usage, including projects, attachments, chat files, chat messages, and artifacts, for misconfigurations and sensitive data. It leverages read-only endpoints for Claude Enterprise to retrieve conversation content and supports Claude Platform for member and workspace changes, API key creation, and file events. The integration enables security teams to triage, assign, and remediate Claude-specific risks using existing workflows and can trigger Cloudflare Gateway policies for automated blocking or restriction of actions.

May 2026Project Glasswing: what Mythos showed usThis post details Cloudflare's experience using Anthropic's Mythos Preview LLM for AI-assisted vulnerability research. It highlights Mythos's capabilities in exploit chain construction and proof generation, contrasting it with general-purpose models. The post also discusses challenges with model refusals in legitimate research, the signal-to-noise problem in vulnerability discovery, and the limitations of using generic coding agents for this task. It introduces the concept of a 'harness' to manage LLM execution for vulnerability discovery, detailing a multi-stage process involving narrow scoping, adversarial review, and parallel task execution.

Apr 2026Building the agentic cloud: everything we launched during Agents Week 2026This post announces the launch of Cloudflare's 'Agents Week 2026,' marking a significant expansion into the 'agentic cloud.' It details numerous new products and features designed to support AI agents, including compute environments (Sandboxes, Artifacts, Durable Objects in Dynamic Workers), security enhancements (Cloudflare Mesh, Managed OAuth for Access), an agent toolbox (Project Think, voice pipeline, Email Service, AI inference layer, Agent Memory, AI Search, Browser Run), and tools for production deployment (CLI, Agent Lee, Flagship, Registrar API). It also addresses the 'agentic web' with an Agent Readiness score and Redirects for AI Training, and provides a network performance update.

Apr 2026Orchestrating AI Code Review at scaleThis post details the architecture and implementation of an AI-powered code review system at Cloudflare. It describes the use of a CI-native orchestration system built around OpenCode, employing specialized AI agents for security, performance, code quality, documentation, release management, and compliance. The system utilizes a plugin architecture for flexibility, processes output via JSONL, and includes mechanisms for real-time processing, error handling, and user feedback to manage LLM thinking time. The core contribution is the engineering approach to integrating LLMs into the critical path of the CI/CD pipeline for automated code review.

Apr 2026The AI engineering stack we built internally — on the platform we shipThis post details the internal engineering stack built on Cloudflare's AI products (AI Gateway, Workers AI, Cloudflare Access, Dynamic Workers, Workflows, Agents SDK) to power AI coding tools for Cloudflare employees. It describes the architecture for authentication, routing, inference, and agent execution, including the use of a proxy Worker for centralized control and the integration of Backstage for knowledge graphs. Key technical contributions include the design of a single discovery endpoint for automatic configuration, the use of JWTs and D1 for anonymous user tracking and cost attribution, and the implementation of a CI integration for AI code review. The post quantifies the internal adoption and impact of these AI tools on developer velocity.

Apr 2026Introducing the Agent Readiness score. Check to see if your site is agent-readyIntroduces the concept of "agent readiness" for websites, providing a new tool (isitagentready.com) and a dataset in Cloudflare Radar to measure adoption of AI agent standards. Details specific standards for discoverability, content accessibility, bot access control, and capabilities, and explains how websites can implement them. The post also highlights Cloudflare's own efforts to make its developer documentation agent-friendly.

Apr 2026Agents that remember: introducing Agent MemoryIntroduced Agent Memory, a managed service for AI agents that provides persistent memory. This service addresses context rot by extracting information from agent conversations during compaction and storing it for future retrieval. It offers an opinionated API for ingesting conversations, remembering specific facts, recalling relevant information, listing memories, and forgetting outdated ones. The ingestion pipeline includes deterministic ID generation, a multi-stage extraction process (full and detail passes), and verification against source transcripts. Agent Memory integrates with Cloudflare Workers via bindings or a REST API and can be used by individual agents, custom agent harnesses, and for shared memory across teams and tools.

Apr 2026Introducing Flagship: feature flags built for the age of AIThis post introduces Flagship, Cloudflare's native feature flag service, specifically designed to support the increasing use of AI in code generation and autonomous agentic workflows. It highlights how feature flags are crucial for safely enabling AI agents to deploy, test, and iterate on code in production by controlling the blast radius. Flagship is built on Cloudflare's infrastructure (Workers, KV, Durable Objects) for edge performance and integrates with the OpenFeature standard for portability.

Apr 2026Unweight: how we compressed an LLM 22% without sacrificing qualityIntroduced Unweight, a lossless compression system for LLM model weights that achieves 15-22% size reduction by compressing only the exponent bytes of BF16 weights using Huffman coding. Implemented four execution pipelines (full decode, exponent-only decode, palette transcode, direct palette) to optimize decompression and inference based on workload characteristics and batch sizes. Decompression occurs in fast on-chip shared memory to avoid HBM bandwidth bottlenecks. Open-sourced GPU kernels and published a technical paper.

Apr 2026Cloudflare’s AI Platform: an inference layer designed for agentsThis post introduces the ability to access third-party AI models (OpenAI, Anthropic, Alibaba Cloud, Google, etc.) through the same `AI.run()` binding as Workers AI, with REST API support coming soon. It also details the ability to bring your own models (containerized with Cog) to Workers AI and highlights automatic failover for model providers and resilient streaming inference for long-running agents. The integration of Replicate's team and technology is also noted.

Apr 2026Building the foundation for running extra-large language modelsThis post details the foundational work on hardware configurations and inference engine optimizations to support extra-large language models, including disaggregated prefill/decode, prompt caching, KV-cache optimization across GPUs, speculative decoding, and the Infire inference engine with multi-GPU support.

Apr 2026Artifacts: versioned storage that speaks GitThis post introduces Artifacts, a new versioned file system built for agents that speaks Git. It addresses the challenges of scaling source control for AI-driven code generation by enabling programmatic repository creation, management, and access via REST API and Workers API. Artifacts leverages Durable Objects and a custom Zig-based Git server to provide a distributed, versioned storage solution for agent sessions, sandboxes, and any data requiring state tracking, time-travel, and persistence. It also introduces ArtifactFS for faster cloning of large repositories.

Apr 2026AI Search: the search primitive for your agentsThis post introduces AI Search, a new primitive for agents that provides plug-and-play search capabilities. Key technical contributions include: 1. Hybrid search combining vector and BM25 keyword matching with parallel execution and result fusion. 2. Built-in storage and indexing for new instances, eliminating the need for external setup. 3. Dynamic instance creation and management via the `ai_search_namespaces` binding in Workers, allowing for per-agent or per-customer instances. 4. Support for metadata attachment to documents for query-time ranking. 5. The ability to query across multiple AI Search instances simultaneously. The post also details the implementation of a customer support agent using AI Search and the Agents SDK, showcasing dynamic instance creation for customer history and the use of `uploadAndPoll` for immediate searchability of saved resolutions.

Apr 2026Project Think: building the next generation of AI agents on CloudflareThis post introduces Project Think, a new set of primitives for the Agents SDK designed to build next-generation AI agents. It addresses limitations of current AI agents by enabling durable execution with fibers, sub-agents via Facets for delegation, persistent sessions with tree-structured messages and FTS5 search, and sandboxed code execution using Dynamic Workers and codemode. This significantly enhances the scalability, cost-effectiveness, and robustness of AI agent deployments by leveraging Cloudflare's infrastructure.

Apr 2026Add voice to your agentIntroduces an experimental voice pipeline for the Agents SDK, enabling real-time voice interaction with AI agents. This includes the `@cloudflare/voice` package with `withVoice` and `withVoiceInput` for full agents and STT-only use cases, respectively. It leverages Workers AI providers (Deepgram Flux, Nova 3, Aura) for STT and TTS, and integrates with the existing Durable Object-based agent architecture for state, persistence, and WebSocket connections. The pipeline supports continuous STT, TTS output, and streaming responses for lower latency. It also introduces `useVoiceAgent` and `useVoiceInput` hooks for React apps and a `VoiceClient` for framework-agnostic clients.

Apr 2026Browser Run: give your agents a browserThis post announces the renaming of 'Browser Rendering' to 'Browser Run' and introduces significant new features specifically designed for AI agents. These include Live View for real-time observation, Human in the Loop for agent assistance, direct Chrome DevTools Protocol (CDP) endpoint for enhanced control and broader language support, MCP client support for AI coding assistants, WebMCP for agent-friendly website interactions, session recordings for debugging, and a substantial increase in concurrent browser limits. These additions position Browser Run as a primary tool for AI agents to interact with the web.

Apr 2026Introducing Agent Lee - a new interface to the Cloudflare stackIntroduces Agent Lee, an in-dashboard AI assistant that acts as a new interface to the Cloudflare stack. Agent Lee uses Codemode to convert Cloudflare MCP tools into a TypeScript API, allowing LLMs to write code for interacting with the platform. It utilizes a Durable Object as a credentialed proxy for sandboxed execution and implements an elicitation gate for write operations, requiring explicit user approval. Agent Lee supports natural language queries for troubleshooting, applying changes, and deploying resources, and generates dynamic UI components like charts and architecture maps.

Apr 2026Scaling MCP adoption: Our reference architecture for simpler, safer and cheaper enterprise deployments of MCPThis post details the implementation of a security architecture for enterprise-wide adoption of Model Context Protocol (MCP) and agentic workflows. It introduces two new concepts: Code Mode with MCP server portals to reduce token costs by enabling progressive tool disclosure and dynamic JavaScript execution, and the use of Cloudflare Gateway for Shadow MCP detection to identify unauthorized remote MCP servers. The post also describes the deployment of remote MCP servers on Cloudflare's developer platform for better visibility and control, and the integration of Cloudflare Access for authentication and authorization of MCP servers. MCP server portals are presented as a solution for discovery, centralized logging, policy enforcement, and data loss prevention.

Apr 2026Agents have their own computers with Sandboxes GAThis post announces the General Availability of Cloudflare Sandboxes and Cloudflare Containers, marking a significant step in enabling AI agents to develop and run code in secure, isolated environments. It details new features like secure credential injection, PTY support for real-time terminals, persistent code interpreters, background processes with live preview URLs, filesystem watching, and snapshots for quick state restoration. The post highlights how these capabilities address key challenges in agentic workloads, such as burstiness, state management, and security, and showcases early adoption by partners like Figma.

Apr 2026Dynamic, identity-aware, and secure Sandbox authThis post introduces outbound Workers for Cloudflare Sandboxes and Containers, enabling programmatic egress proxies for secure connectivity, observability, and flexible authentication for AI agents. It details how these outbound Workers can enforce zero-trust principles, inject credentials dynamically, integrate with other Cloudflare developer platform services (R2, KV), and provide dynamic networking controls, significantly enhancing the security and manageability of agentic workloads.

Mar 2026Powering the agents: Workers AI now runs large models, starting with Kimi K2.5This post announces the integration of large open-source models, specifically Kimi K2.5, into Workers AI, enabling cost-effective and high-performance inference for agentic workloads. It details the underlying inference stack optimizations, including custom kernels and parallelization techniques. New platform features for agentic workloads include prefix caching with surfaced metrics and discounts, a new `x-session-affinity` header for improved cache hit rates, and a redesigned asynchronous API for durable inference. The post quantifies cost savings (77% reduction for a specific use case) and performance benefits (faster TTFT, higher TPS) derived from these advancements.

Mar 2026Slashing agent token costs by 98% with RFC 9457-compliant error responsesIntroduced RFC 9457-compliant structured Markdown and JSON error payloads for AI agents, replacing HTML error pages. This provides machine-readable instructions, reduces payload size and token usage by over 98% for common errors like rate limiting, and offers stable fields for agent control flow (e.g., retryable, retry_after, owner_action_required). The implementation covers all 1xxx-class errors and will extend to 4xx/5xx errors, providing a consistent network-wide contract for agentic web interactions.

Mar 2026AI Security for Apps is now generally availableAI Security for Apps is now generally available, providing threat detection and mitigation for AI-powered applications. Key new features include free AI endpoint discovery for all plans, custom topic detection for tailored security policies, and improved prompt extraction mechanisms. The service integrates with the WAF rule builder for mitigation and offers enhanced visibility into AI deployments.

Mar 2026From reactive to proactive: closing the phishing gap with LLMsThis post details the application of LLMs to email security, moving from reactive detection based on user-reported misses to proactive identification of emerging threats by analyzing large volumes of email data. LLMs are used to categorize emails with high fidelity, enabling the creation of targeted ML models for specific threat classes like 'Sales Outreach' phishing. This approach significantly reduces false positives and improves the speed of detection model updates.

Mar 2026How Cloudy translates complex security into human actionThis post introduces the integration of Cloudy, an LLM-powered explanation layer, into Cloudflare One. Cloudy translates complex machine learning outputs into human-readable guidance for security teams and end users. It is being extended to Cloudflare Email Security and CASB. For Email Security, Cloudy provides explanations within the Phishnet tool to help end-users understand why an email was flagged, reducing unnecessary escalations to the SOC. This is achieved by aggregating structured outputs from multiple detection models and passing them to Workers AI for natural language explanation generation. For CASB, Cloudy translates complex findings into structured explanations for administrators, detailing the rationale behind detections and providing specific match details. The implementation leverages Cloudflare Workers AI for real-time generation of these summaries.

Feb 2026Code Mode: give agents an entire API in 1,000 tokensIntroduced a new Model Context Protocol (MCP) server for the entire Cloudflare API that leverages 'Code Mode' to significantly reduce token usage. This server provides access to over 2,500 API endpoints through just two tools, `search()` and `execute()`, enabling AI agents to interact with Cloudflare services efficiently. Also open-sourced a Code Mode SDK for developers to implement similar patterns.

Feb 2026Introducing Markdown for AgentsIntroduces 'Markdown for Agents', a feature that allows AI agents to request content in markdown format via the `Accept: text/markdown` header. Cloudflare's network intercepts these requests, converts the HTML origin content to markdown on the fly, and serves it. This reduces token usage for AI processing by up to 80%. The feature includes an `x-markdown-tokens` header for estimating token count and `Content-Signal` headers for expressing content usage preferences. Cloudflare Radar now tracks content type insights for AI bot traffic, including markdown usage.

Jan 2026Introducing Moltworker: a self-hosted personal AI agent, minus the minisThis post introduces Moltworker, a middleware Worker and adapted scripts that allow running the self-hosted AI agent Moltbot on Cloudflare's Sandbox SDK and Developer Platform APIs. It highlights the advancements in Node.js compatibility within Cloudflare Workers, the integration of AI Gateway for managing AI provider secrets and billing, the use of Sandboxes for secure execution of untrusted code, R2 for persistent storage, and Browser Rendering for web automation, demonstrating how Cloudflare's developer platform can host complex AI applications.

Dec 2025Why Replicate is joining CloudflareThis post announces the acquisition of Replicate by Cloudflare. It details Replicate's journey in making AI models accessible to developers, starting with Cog for packaging and Replicate.com for serving. It highlights the evolution of AI engineering beyond just model inference to include microservices, object storage, caching, and databases. The acquisition is framed as a strategic move to combine Replicate's model primitives with Cloudflare's network, Workers, R2, and Durable Objects to build a comprehensive AI infrastructure layer, enabling edge inference and model pipelines on Workers.

Nov 2025Partnering with Black Forest Labs to bring FLUX.2 -dev- to Cloudflare Workers AIThis post introduces the FLUX.2 [dev] model to Cloudflare Workers AI, detailing its capabilities for photorealistic image generation, character consistency through multi-reference inputs, and granular controls like JSON prompting and hex code color specification. It provides code examples for using the model via curl and Workers AI Binding, highlighting its use cases for ad variations, product shots, and dynamic editorials. The post also mentions the underlying technical implementation of multipart form data for image inputs and output.

Nov 2025Replicate is joining CloudflareThis post announces the acquisition of Replicate by Cloudflare, integrating Replicate's model catalog and developer community with Cloudflare's AI Cloud. Key technical contributions include: enabling the execution of Replicate's 50,000+ models on Cloudflare's Workers AI platform, bringing fine-tuning capabilities to Workers AI, and supporting the deployment of custom models packaged with Replicate's Cog tool. The integration also extends to the AI Gateway for unified observability, prompt management, A/B testing, and cost analytics across all AI inference traffic.

Oct 2025Beyond IP lists- a registry format for bots and agentsProposes a registry format for bots and agents to discover public keys, extending the Web Bot Auth protocol. Outlines a lightweight format for a list of URLs pointing to agent keys and a complementary 'signature-agent card' format for additional metadata. Demonstrates integration with Caddy server and mentions Cloudflare's intent to provide an initial registry.

Oct 2025Policy, privacy and post-quantum: anonymous credentials for everyoneThis post introduces the concept of Anonymous Credentials (ACs) and their application to privacy-preserving rate-limiting for AI agentic platforms and users. It details the requirements for ACs, including blind signatures, nullifiers, and public attributes, and discusses the challenges of migrating to post-quantum (PQ) cryptography for these primitives. The post highlights the current state of classical AC adoption in initiatives like the EU's digital identity wallet and ongoing IETF efforts, while also outlining the need for PQ-unlinkable and unforgeable ACs to address future threats.

Oct 2025Securing agentic commerce: helping AI Agents transact with Visa and MastercardIntroduces Visa's Trusted Agent Protocol and Mastercard's Agent Pay, which leverage Web Bot Auth and HTTP Message Signatures to authenticate AI agents for commerce. Details the validation process performed by Cloudflare, including signature verification, nonce uniqueness checks, and tag validation (agent-browser-auth vs. agent-payer-auth). Outlines the future integration with Cloudflare's Agent SDK and the creation of managed rulesets for merchants.

Sep 2025An AI Index for all our customersThis post introduces AI Index, a new capability that automatically creates AI-optimized search indexes for customer domains. It provides APIs for AI builders to discover and retrieve web content, including MCP server, search API, LLMs.txt, bulk data API, and pub/sub subscriptions. It also introduces the Open Index, an aggregated layer of participating AI Indexes. Monetization options like Pay per crawl and x402 integrations are highlighted. The system leverages existing AI Search technology for index creation and AI Crawl Control for access management.

Sep 2025Code Mode: the better way to use MCPIntroduces 'Code Mode' for AI agents, which transforms MCP tools into a TypeScript API. This allows LLMs to generate and execute TypeScript code that calls these APIs, improving their ability to handle complex tools and multi-step operations. The Cloudflare Agents SDK is updated to support this, fetching MCP schemas and converting them into TypeScript definitions for use within a sandboxed execution environment.

Sep 2025Choice: the path to AI sovereigntyThis post introduces the availability of three new regional AI models on Cloudflare Workers AI: IndicTrans2 (for Indic languages), PLaMo-Embedding-1B (Japanese text embedding), and SEA-LION v4-27B (Southeast Asian languages). It details how these models support AI sovereignty by enabling local language AI development and deployment at the edge. The post provides curl examples for using these models, demonstrating their integration into the Workers AI platform for inference.

Sep 2025Giving users choice with Cloudflare’s new Content Signals PolicyIntroduced the Content Signals Policy, a new addition to robots.txt that allows website operators to express preferences for how their content can be used after access, specifically defining signals for 'search', 'ai-input', and 'ai-train'. This policy aims to address the free-rider problem and the choice between open access and data lockdown for AI training and usage.

Sep 2025To build a better Internet in the age of AI, we need responsible AI bot principles. Here’s our proposal.This post proposes five responsible AI bot principles: public disclosure, self-identification (with a future emphasis on cryptographic verification via Web Bot Auth), declared single purpose, respect for preferences, and acting with good intent. It details the rationale behind these principles, highlighting the impact of AI bots on web publishers and the need for consensus among stakeholders. It also references Cloudflare's existing efforts in bot management and AI crawler policies.

Sep 2025Building unique, per-customer defenses against advanced bot threats in the AI eraIntroduces a new platform for per-customer behavioral anomaly detection against AI-driven bots. This platform builds unique detection suites for each bot management customer by establishing dynamic baselines of normal traffic, identifying deviations from these norms, and generating actionable findings. It leverages machine learning models and integrates with existing Cloudflare security features, including new Bot Detection IDs for WAF rules and enhanced ML-based detection for residential proxy networks.

Sep 2025Cloudflare Confidence Scorecards - making AI safer for the InternetIntroduces Cloudflare Application Confidence Scorecards, a new feature within the Cloudflare One SASE platform that automates the evaluation of AI and SaaS applications. The scorecards provide a quantitative assessment of application trustworthiness based on security, privacy, data management, financial stability, and AI-specific risks (compliance, deployment security, system card, training data governance). The system automates data collection by crawling public resources and uses LLMs for parsing, with human validation for accuracy. The scores are integrated into the Application Library for visibility and will soon be used for enforcement policies within Cloudflare Gateway and DLP.

Sep 2025Deploy your own AI vibe coding platform — in one click!This post introduces VibeSDK, an open-source platform for building and deploying AI-powered development platforms. It details the architecture and implementation of a one-click AI vibe coding platform, leveraging Cloudflare Sandboxes for isolated development environments, the Agents SDK for LLM integration, R2 for storing project templates, and Workers for Platforms for scalable deployment. It also emphasizes the use of AI Gateway for multi-model support, response caching, and observability of AI inference traffic.

Sep 2025Helping protect journalists and local news from AI crawlers with Project GalileoThis post introduces the integration of Cloudflare's Bot Management and AI Crawl Control services into Project Galileo, offering free protection against AI crawlers for journalists and non-profits. It details how AI Crawl Control identifies and categorizes AI crawlers, provides data on their activity, and allows website owners to control access, enabling content protection and negotiation for fair compensation.

Sep 2025Launching the x402 Foundation with Coinbase, and support for x402 transactionsIntroduces the x402 protocol for machine-to-machine payments, enabling AI agents to pay for resources. Proposes a new deferred payment scheme for x402 to support scenarios like pay-per-crawl and batch settlements. Integrates x402 support into the Agents SDK and MCP servers, demonstrated with a playground and example code for agentic payments and tool usage.

Sep 2025AI Week 2025: RecapThis post is a recap of Cloudflare's AI Week 2025, announcing numerous new products, partnerships, and features related to AI. Key technical contributions include: advancements in AI security (prompt protection, application confidence scores, CASB scanning for LLMs, MCP Server Portals, AI-SPM best practices), AI bot traffic analysis and control (AI-aware caching, Web Bot Auth, AI Crawl Control, traffic analysis by purpose), developer tools for building AI experiences (AI Gateway enhancements, Infire inference engine, Workers AI model expansion, Omni inference platform, Firewall for AI, real-time voice agents), and AI-powered improvements to Cloudflare's internal operations (image segmentation model evaluation, Cloudy for threat analysis and email detection summarization, AI-powered network troubleshooting).

Aug 2025Automating threat analysis and response with CloudyThis post details the integration of Cloudflare's AI agent, Cloudy, with its security analytics functionality and introduces a new conversational interface for faster root cause analysis and mitigation. It also describes the integration of Cloudy with the Cloudforce One Threat Events platform, allowing users to query global threat data in natural language. The technical implementation leverages the Agents SDK and Workers AI, with a commitment to not training on customer data.

Aug 2025Cloudflare is the best place to build realtime voice agentsIntroduces Cloudflare Realtime Agents, a runtime for orchestrating voice AI pipelines at the edge. Enables piping raw WebRTC audio as PCM to Workers via WebSockets, facilitating real-time AI inference. Adds WebSocket support to Workers AI for models like PipeCat's smart-turn-v2. Integrates Deepgram for speech-to-text and text-to-speech within Workers AI.

Aug 2025Cloudy Summarizations of Email Detections: Beta AnnouncementThis post introduces the beta of Cloudy email detection summaries, an AI-powered feature that translates complex email threat detection logic into clear, human-readable explanations for Security Operations Center (SOC) teams. It details the challenge of opaque detection rules, the hallucination problem encountered during initial LLM implementation, and the safeguards implemented (RAG and model context enhancement) to mitigate this risk. The post provides an example of an improved summary for the SPAM.ASNReputation.IPReputation_Scuttle.Anomalous_HC rule.

Aug 2025The crawl-to-click gap: Cloudflare data on AI bots, training, and referralsThis post analyzes the shift in AI bot traffic patterns, noting a significant increase in training-related crawling and a corresponding decrease in referral traffic from search engines to news sites. It quantifies the growth of specific AI crawlers (GPTBot, ClaudeBot, Meta-ExternalAgent) and the decline of others (Amazonbot, Bytespider). The analysis introduces the 'crawl-to-refer ratio' to illustrate the imbalance between bot activity and user referrals, providing specific metrics for various AI platforms. The post also details the growth of AI-only crawlers and the overall trend of AI and search crawling, including seasonal fluctuations.

Aug 2025A deeper look at AI crawlers: breaking down traffic by purpose and industryThis post introduces new capabilities on Cloudflare Radar's AI Insights page, providing industry-focused AI bot traffic data and a breakdown of AI bot traffic by purpose (Training, Search, User action, Undeclared). It details how these new filters and breakdowns allow users to analyze AI crawler activity, compare it across industries, and understand crawl-to-refer ratios.

Aug 2025Evaluating image segmentation models for background removal for ImagesThis post details the evaluation and selection process for image segmentation models to implement a background removal feature within Cloudflare Images. It introduces image segmentation concepts, explains accuracy metrics (Intersection over Union, Dice coefficient, pixel accuracy), and compares the performance of U2-Net, IS-Net, BiRefNet, and SAM models based on these metrics. The technical contribution lies in the methodology for evaluating and selecting AI models for a specific product capability, highlighting the trade-offs between different model architectures and their suitability for real-world applications.

Aug 2025Make Your Website Conversational for People and Agents with NLWeb and AutoRAGThis post introduces the integration of AutoRAG with the NLWeb standard to enable conversational search on websites. It details the architectural changes within AutoRAG to support website crawling and indexing, including the introduction of JobManager and FileManager Durable Objects to handle large, dynamic data sources and improve scalability. The post also describes the deployment of a Cloudflare Worker that implements the NLWeb protocol, providing both a conversational UI and an MCP server endpoint for AI agents. This represents a significant expansion of AutoRAG's capabilities to treat websites as first-class data sources.

Aug 2025The age of agents: cryptographically recognizing agent trafficIntroduces 'signed agents' as a new classification for AI agent traffic, extending the verified bots program. This involves using HTTP message signatures (Web Bot Auth) to cryptographically verify agent requests, allowing website owners to granularly control access. The post details the policy for signed agents, the submission process via the Cloudflare dashboard, and the integration of this classification into security rules and Cloudflare Radar. It also highlights partnerships with initial signed agents like ChatGPT agent, Goose, Browserbase, and Anchor Browser.

Aug 2025The next step for content creators in working with AI bots: Introducing AI Crawl ControlIntroduces AI Crawl Control (formerly AI Audit) to general availability, rebranding it to emphasize control over AI crawler access. Details the use of HTTP 402 ('Payment Required') status codes to facilitate licensing agreements between content creators and AI crawlers, allowing customization of messages for communication and commercial terms. Outlines plans for future enhancements to include structured data on content value, freshness, and licensing terms within 402 responses, and mentions the ongoing beta of 'pay per crawl' for automated monetization.

Aug 2025AI Gateway now gives you access to your favorite AI models, dynamic routing and more — through just one endpointThis post introduces several new features for Cloudflare AI Gateway: unified billing across multiple AI providers (Anthropic, Google, Groq, OpenAI, xAI) via Cloudflare credits, secure storage and distribution of provider API keys using Cloudflare Secrets Store with granular RBAC and audit logging, an automatic translation layer for unified request/response handling between providers, dynamic routing capabilities for AI traffic based on request attributes (e.g., rate limiting, A/B testing, model chaining), and integrated Data Loss Prevention (DLP) scanning within the AI Gateway Firewall for enhanced security. The post also details the integration with Secrets Store for managing BYO Provider Keys and the use of the 'AI Gateway' scope for secrets.

Aug 2025How Cloudflare runs more AI models on fewer GPUs: A technical deep-diveThis post details the internal platform 'Omni' designed to improve GPU utilization for AI models on Cloudflare's edge nodes. Omni achieves this by: 1. Spawning multiple models from a single control plane (scheduler) using lightweight process isolation and Python virtual environments managed by uv. 2. Implementing a virtual filesystem using FUSE for /proc/meminfo to accurately reflect per-model memory limits, preventing OOM errors. 3. Over-committing GPU memory by injecting a CUDA stub library that intercepts memory allocation calls and forces them into unified memory mode, allowing more models to share a GPU at the cost of potential latency during memory transfers. 4. Enforcing memory limits by overriding CUDA runtime APIs to expose only a subset of memory to each model. Omni integrates with Workers AI, KV, and various inference engines (vLLM, Infire) to provide a unified layer for model management and inference.

Aug 2025How we built the most efficient inference engine for Cloudflare’s networkThis post introduces Infire, a new LLM inference engine written in Rust, designed to optimize GPU, memory, and network I/O utilization for Cloudflare's edge network. It details architectural challenges of running LLM inference at the edge, including the need for efficient resource utilization, dynamic workload handling, and strong security sandboxing. Infire employs continuous batching with chunked prefill and optimized KV cache handling. It also utilizes Page Locked memory with CUDA asynchronous copy over multiple streams for faster model loading onto GPUs. The engine is currently powering the Llama 3.1 8B model for Workers AI.

Aug 2025State-of-the-art image generation Leonardo models and text-to-speech Deepgram models now available in Workers AIThis post announces the integration of Leonardo.Ai image generation models (@cf/leonardo/phoenix-1.0 and @cf/leonardo/lucid-origin) and Deepgram speech-to-text (@cf/deepgram/nova-3) and text-to-speech (@cf/deepgram/aura-1) models into Cloudflare Workers AI. It provides example cURL and JavaScript API calls for using these models, including details on image generation parameters (width, height, steps, seed, guidance) and audio processing (contentType, detect_language). The post also mentions the addition of WebSocket support for Deepgram models and the ability to send binary mp3 files directly. It emphasizes the synergy of these models with other Cloudflare Developer Platform products like Workers, R2, Images, and Realtime for building comprehensive AI applications.

Aug 2025Securing the AI Revolution: Introducing Cloudflare MCP Server PortalsIntroduces Cloudflare MCP Server Portals, a new capability to centralize, secure, and observe MCP connections. MCP Server Portals act as a single gateway for MCP servers, integrating with Cloudflare One for policy enforcement (MFA, device posture, geo-restriction), comprehensive visibility and logging of MCP traffic, and simplified user configuration by providing a single Portal endpoint. It enables administrators to review and approve MCP servers, enforcing least privilege. For MCP servers hosted on Cloudflare, Access policies can enforce direct authorization via linked OAuth applications. For external servers, authorization controls are managed via OAuth. Future plans include additional enforcement mechanisms for MCP Servers, WAF integration for AI traffic, Cloudflare-hosted MCP Servers via AI Gateway, ML-based abuse detection, and MCP standard enhancements.

Aug 2025Best Practices for Securing Generative AI with SASEThis post introduces new AI Security Posture Management (AI-SPM) features built into Cloudflare's SASE platform to address the security challenges of generative AI adoption. These features include shadow AI reporting for visibility into unsanctioned AI use, confidence scoring for AI providers, AI prompt protection against malicious inputs, out-of-band API CASB integrations for detecting misconfigurations, and tools to secure Model Context Protocol (MCP) deployments. The post outlines best practices for developing an AI Security Strategy and leveraging Cloudflare's SASE platform for visibility, risk management, and data protection in the context of AI.

Aug 2025Block unsafe prompts targeting your LLM endpoints with Firewall for AIThis post introduces unsafe content moderation integrated directly into Cloudflare Firewall for AI, leveraging Llama Guard 3 for real-time detection and blocking of harmful prompts at the edge. It details the asynchronous architecture for scaling detection modules, the use of Workers AI for Llama Guard inference, and the enforcement of policies through custom rules. Future enhancements will include prompt injection and jailbreak detection.

Aug 2025ChatGPT, Claude, & Gemini security scanning with Cloudflare CASBThis post introduces API-based Cloud Access Security Broker (CASB) integrations for ChatGPT, Claude, and Gemini. These integrations allow users to connect their GenAI accounts to Cloudflare One and scan for security issues such as misconfigurations, Data Loss Prevention (DLP) matches, data exposure, and compliance risks. For ChatGPT, it detects capability activation (actions, code execution, web access), external exposure of chats and GPTs, secrets/keys/invites, and sensitive content via DLP. For Claude, it surfaces secrets/keys/invites and sensitive content via DLP. For Gemini, it detects identity/MFA issues and license hygiene within Google Workspace. These integrations provide agentless, out-of-band visibility into the security posture and data risks within these GenAI tools.

Aug 2025Introducing Cloudflare Application Confidence Score For AI ApplicationsThis post introduces Cloudflare Application Confidence Scores, a new system for evaluating the safety, security, and data protection of SaaS and Gen AI applications. It details the design principles, the two distinct scoring categories (Application Confidence and Gen-AI Confidence), and the specific criteria within the rubric for each score. The rubric is based on publicly available data and aims for transparency and objectivity, with human oversight for accuracy. The post provides examples of how different tiers of OpenAI's ChatGPT are scored based on their data handling and security practices.

Aug 2025Beyond the ban: A better way to secure generative AI applicationsThis post introduces AI prompt protection as a new capability within Cloudflare's Data Loss Prevention (DLP) product, integrated into Cloudflare One. It details the technical components of this feature: prompt detection (capturing user prompts and AI responses from web-based AI tools), topic classification (analyzing content and intent of prompts into categories like PII, Credentials, Source Code, Customer Data, Financial Information, Code Abuse, Jailbreak), guardrails (enabling granular policies based on classified topics, allowing identity-aware controls), and logging (capturing prompts and responses with customer-provided encryption for auditability and incident investigation). The post also mentions the acquisition of Kivera to enhance operation mapping for API analysis, which aids in detecting prompts.

Aug 2025Unmasking the Unseen: Your Guide to Taming Shadow AI with Cloudflare OneThis post introduces 'Shadow AI' as a new security blind spot, detailing how employees use unsanctioned AI tools, leading to potential data leaks and intellectual property exposure. It announces the Cloudflare Shadow IT Report, enhanced with AI service classification and powered by TimescaleDB for improved analytics. The post outlines how Cloudflare One, specifically Cloudflare Gateway, can be used to proxy traffic, provide granular visibility into AI application usage, allow for application approval status management (Approved, Unapproved, In Review), and enforce policies such as blocking unapproved AI, applying Data Loss Prevention (DLP), restricting specific actions, and utilizing Browser Isolation. It also highlights the use of Cloudflare Log Explorer for deeper forensic investigations.

Aug 2025How we built AI face cropping for ImagesThis post details the engineering effort behind bringing AI face cropping to Cloudflare Images. It describes the transition from a CPU-based prototype to a GPU-based implementation using Workers AI to address scalability and memory leak issues. The technical details include the selection of the RetinaFace model for face detection, its performance characteristics on the WIDERFACE dataset, and the inference process including image resizing to 1024x1024 for efficiency. It also explains how bounding boxes are used to construct an outer bounding box for cropping and the use of the `gravity=face` and `zoom` parameters for image transformation.

Aug 2025Partnering with OpenAI to bring their new open models onto Cloudflare Workers AIThis post announces the integration of OpenAI's new open-weight models (GPT-OSS 120B and 20B) into Cloudflare Workers AI. It details the technical specifications of these Mixture-of-Experts (MoE) models, including their FP4 quantization and support for reasoning, tool calling, Code Interpreter, and upcoming Web Search capabilities. The post outlines the API formats supported for interacting with these models on Workers AI (Responses API, Chat Completions API) and highlights the implementation of stateful Code Interpreter execution using Cloudflare Sandboxes and Containers, with pre-warmed environments for faster startup times.

Jul 2025Control content use for AI training with Cloudflare’s managed robots.txt and blocking for monetized contentIntroduces two new tools for website owners to control AI bot access to their content: a managed robots.txt file service and an option to block AI bots only on monetized portions of a site. This addresses the shift in the crawler relationship from symbiotic to extractive, evidenced by high crawl-to-referral ratios for AI bots. The post also provides updated data on AI bot traffic and usage of Cloudflare's existing AI bot blocking features, highlighting the underutilization of robots.txt for AI crawlers and the increasing adoption of Cloudflare's AI bot management tools.

Jul 2025From Googlebot to GPTBot: Who’s crawling your site in 2025This post analyzes the evolving landscape of web crawlers, with a specific focus on the rise of AI crawlers. It presents data on the changing market share of AI bots like GPTBot, ClaudeBot, and Meta-ExternalAgent, and highlights the significant growth of Googlebot. The post also discusses the implications of AI crawling for content creators and the tools Cloudflare is developing to manage and enforce crawler policies, such as AI Audit.

Jul 2025Introducing pay per crawl: Enabling content owners to charge AI crawlers for accessIntroduces 'pay per crawl' functionality, enabling content owners to charge AI crawlers for access using HTTP 402 Payment Required status codes and authenticated requests. This feature integrates with existing web infrastructure, WAF, and bot management, and includes mechanisms for publishers to define pricing and for crawlers to authenticate and pay via HTTP headers.

Jul 2025The crawl before the fall… of referrals: understanding AI’s impact on content providersThis post introduces a new metric, the 'crawl-to-referral ratio,' to measure the impact of AI bots on content providers. The metric is calculated by dividing the total number of HTML page requests from AI crawlers by the number of HTML page requests from clients referred by AI platforms. It details the methodology using User-Agent strings and Referer headers, presents observational data for various AI platforms (Anthropic, Mistral, Google, DuckDuckGo, Yandex, Baidu, OpenAI's GPTBot), and discusses trends in crawling and referral traffic. It also highlights existing tools for blocking AI crawlers and introduces a new concept of 'pay-per-crawl' to ensure fair value exchange.

Jun 2025Connect any React application to an MCP server in three lines of codeThis post introduces `use-mcp`, a React library for building remote MCP clients, and open-sources the AI Playground. `use-mcp` abstracts away transport protocols (Streamable HTTP, SSE), authentication (OAuth 2.1), session management, and dynamic tool discovery. It provides connection state management, retry logic with backoff, and a debug log array. The AI Playground, built with Workers AI and `use-mcp`, offers a chat interface for interacting with MCP servers and LLMs, with built-in debugging and monitoring of MCP connections and tool invocations.

May 2025Thirteen new MCP servers from Cloudflare you can use todayThis post announces the release of thirteen new remote Model Context Protocol (MCP) servers that Cloudflare is making publicly available. These servers expose various Cloudflare services and data to MCP clients, enabling AI agents to interact with Cloudflare's ecosystem. The servers include: Cloudflare Documentation, Workers Bindings, Workers Observability, Container, Browser Rendering, Radar, Logpush, AI Gateway, AutoRAG, Audit Logs, DNS Analytics, Digital Experience Monitoring, and Cloudflare One CASB. The post details the functionality of each server and provides links to their respective GitHub repositories and documentation. It also highlights Cloudflare's collaboration with Anthropic and other companies that have built remote MCP servers on Cloudflare.

May 2025MCP Demo Day: How 10 leading AI companies built MCP servers on CloudflareThis post introduces the Model Context Protocol (MCP) and demonstrates its adoption by 10 leading AI companies to build remote MCP servers on Cloudflare. It highlights Cloudflare's tooling for simplifying MCP server development, including one-click deployment, support for Python and Streamable HTTP transport, and best practices. The post showcases specific use cases from Asana, Atlassian, and Intercom, detailing how they leverage MCP to connect their services to AI agents for enhanced user experiences, personalized interactions, and easier integration with their existing workflows. The core technical contribution is the enablement and demonstration of remote MCP server deployment on Cloudflare's platform, abstracting away protocol complexity, authentication, and transport.

Apr 2025Bringing streamable HTTP transport and Python language support to MCP serversIntroduced Streamable HTTP transport for MCP, allowing bidirectional communication over a single endpoint and dynamic upgrades to SSE for streaming responses. Added Python language support for building MCP servers on Cloudflare Workers using the Python MCP SDK and FastAPI-MCP. Provided code examples for configuring MCP servers to support both SSE and Streamable HTTP transports concurrently, and for defining tools and resources in Python MCP servers.

Apr 2025Startup spotlight: building AI agents and accelerating innovation with Cohort #5This post details how startups Lamatic.ai and Skyward AI are leveraging Cloudflare's platform to build and scale AI agent infrastructure. Lamatic.ai uses Workers for per-customer deployment, Workers KV for encrypted credential storage, and Queues for orchestrating ETL and event-driven workflows. Skyward AI utilizes Workers, Durable Objects, and the new AI agents framework for orchestrating compliance agents, AI Gateway for observability, Hyperdrive for database query latency reduction, and R2 for long-term storage. The post highlights the use of Workers for Platforms to enable customer-specific serverless functions and the benefits of the new AI agents framework in simplifying agent communication and state management.

Apr 2025Workers AI gets a speed boost, batch workload support, more LoRAs, new models, and a refreshed dashboardThis post details significant engineering advancements in Cloudflare's Workers AI platform. Key contributions include the implementation of speculative decoding for 2-4x faster inference, leveraging unused GPU compute by running a smaller draft model concurrently with the main model. Prefix caching has been introduced to reduce 'pre-fill' time for requests reusing context, benefiting RAG, code generation, and chatbots. An asynchronous batch API has been launched to handle large workloads that are not latency-sensitive but require completeness guarantees, preventing immediate errors due to capacity constraints and improving overall inference server utilization. LoRA support has been expanded to 8 models with larger ranks (up to 32) and file sizes (up to 300MB). The post also mentions the addition of new models and a refreshed dashboard for usability.

Apr 2025Introducing Cloudflare Secrets Store (Beta): secure your secrets, simplify your workflowIntroduces Cloudflare Secrets Store (Beta) as a centralized platform for managing secrets like API tokens and credentials across Cloudflare products, starting with Cloudflare Workers. Details the security architecture involving DEKs and KEKs, RBAC for access control, and integration with Quicksilver for network-wide propagation. Highlights future plans to support more products like Cloudflare Access, Transform Rules, and AI Gateway, and to allow binding to the store itself and multiple secret stores per account.

Apr 2025Introducing AutoRAG: fully managed Retrieval-Augmented Generation on CloudflareThis post introduces AutoRAG, a fully managed Retrieval-Augmented Generation (RAG) pipeline on Cloudflare. It details the end-to-end process of building a RAG system, from data ingestion and automatic chunking/embedding to storing vectors in Vectorize, performing semantic retrieval, and generating responses using Workers AI. The indexing process involves file ingestion from data sources (initially R2), Markdown conversion using Workers AI, chunking, embedding with Workers AI's embedding model, and storage in Vectorize. The querying process includes receiving queries, optional query rewriting, embedding the query, vector search in Vectorize, metadata and content retrieval, and response generation using a Workers AI text-generation model. A tutorial demonstrates using a Cloudflare Worker with Puppeteer and Browser Rendering API to fetch webpages, render them, and upload the HTML content to an R2 bucket for AutoRAG processing.

Apr 2025Piecing together the Agent puzzle: MCP, authentication & authorization, and Durable Objects free tierThis post introduces new capabilities for AI agents to act as remote MCP clients, with built-in transport (SSE, HTTP, Streamable HTTP) and connection management. It also details integrations with Stytch, Auth0, and WorkOS for BYO OAuth providers for MCP servers, enabling authentication and authorization. Hibernation support for McpAgent is introduced to manage stateful, remote MCP servers. The Durable Objects free tier is now available, and Workflows are generally available for production-ready, long-running, multi-step actions. AutoRAG is also highlighted as a way to integrate context-aware AI.

Apr 2025Meta Llama 4 now available on Workers AICloudflare is now offering Meta's Llama 4 Scout model on its Workers AI platform. Llama 4 Scout is a Mixture of Experts (MoE) model with 109B total parameters and 17B active parameters, featuring a novel early-fusion architecture that makes it natively multimodal. It supports a context window of up to 10 million tokens, with Workers AI initially supporting 131,000 tokens. The post explains MoE architecture and early-fusion, highlighting how Workers AI handles the infrastructure and memory requirements for hosting such large models serverlessly.

Apr 2025Improve your media pipelines with the Images binding for Cloudflare WorkersIntroduces the Images binding for Cloudflare Workers, enabling programmatic image transformation, resizing, and encoding directly within the Workers runtime. This binding decouples image optimization from delivery, supports local development without charges, and simplifies complex image manipulation workflows by allowing direct manipulation of uploaded image data and chaining of operations like transform, draw, and output. It also demonstrates an example of watermarking and encoding an uploaded image and storing it in R2.

Mar 2025Build and deploy Remote Model Context Protocol (MCP) servers to CloudflareIntroduced the ability to build and deploy remote Model Context Protocol (MCP) servers to Cloudflare. This includes the `workers-oauth-provider` library for OAuth integration, the `McpAgent` class in the Cloudflare Agents SDK for remote transport, the `mcp-remote` adapter for local client compatibility, and an AI playground as a remote MCP client. The post details the MCP OAuth flow, the role of the MCP server as an OAuth Provider, and the benefits of issuing custom tokens for granular access control.

Mar 2025Improving Data Loss Prevention accuracy with AI-powered context analysisIntroduces AI-powered context analysis to Cloudflare's Data Loss Prevention (DLP) solution, leveraging Workers AI for text embedding and Vectorize for vector database storage. This feature aims to reduce false positives by analyzing the context of data matches and learning from customer feedback. It also details the integration with Cloudflare Queues for asynchronous indexing and discusses latency implications and future multilingual support.

Mar 2025Introducing Cloudy, Cloudflare’s AI agent for simplifying complex configurationsThis post introduces Cloudy, Cloudflare's first AI agent, which is embedded within WAF Custom Rules and Cloudflare One Gateway policies. Cloudy leverages Workers AI and LLMs to analyze configurations, providing summaries, identifying redundant/conflicting rules, optimizing execution order, and highlighting disabled rules. It also summarizes Cloudflare One Gateway policies. The LLM is provided with policy/rule data, product documentation, enriched datasets (translating IDs to plain text), and output instructions (JSON). Future plans include expanding AI agent capabilities across other Cloudflare products.

Mar 2025Cloudflare for AI: supporting AI adoption at scale with a security-first approachThis post introduces 'Cloudflare for AI,' a new umbrella offering that integrates and expands upon existing AI-related capabilities. It highlights the development of an AI Agents SDK, the continued investment in serverless AI inference infrastructure, and the use of R2 for AI training data storage. New features include 'Firewall for AI' for prompt injection protection and PII control, and 'AI Crawl Control' (formerly AI Audit) for visibility into AI crawlers accessing content. The post also mentions the integration of Llama Guard into AI Gateway and Firewall for AI for content moderation.

Mar 2025An early look at cryptographic watermarks for AI-generated contentThis post introduces the concept of cryptographic watermarking for AI-generated content, specifically focusing on pseudorandom error-correcting codes. It details the security goals of robustness, undetectability, and unforgeability, and contrasts them with current deep learning-based watermarking techniques. The post outlines the Christ-Gunn-2024 framework for watermarks using pseudorandom codes, which involves key generation, encoding, and decoding algorithms to embed verifiable origin information into AI artifacts.

Mar 2025How we train AI to uncover malicious JavaScript intent and make web surfing saferThis post details the integration of a new AI model into Cloudflare's Page Shield product. This model uses Graph Neural Networks (GNNs) to analyze JavaScript syntax trees and detect specific malicious intents (Magecart, crypto mining, malware) with greater precision than previous methods. This represents a significant advancement in Cloudflare's use of AI for proactive client-side security.

Mar 2025Take control of public AI application security with Cloudflare's Firewall for AIThis post introduces Firewall for AI, an inline security solution that protects user-facing LLM-powered applications. It focuses on three key capabilities: discovering LLM-powered endpoints using heuristics like request path signatures, server-sent events, and origin response bitrate; detecting Personally Identifiable Information (PII) in prompts using a hybrid approach of regex and Named Entity Recognition (NER) models deployed via Workers AI with Presidio; and mitigating risks by integrating with Cloudflare's WAF for policy enforcement. The post details the technical challenges and solutions for discovering LLM endpoints and the architecture for PII detection and mitigation.

Mar 2025Trapping misbehaving bots in an AI LabyrinthIntroduced AI Labyrinth, a new mitigation approach that uses AI-generated content to slow down, confuse, and waste the resources of AI Crawlers and other bots that don't respect "no crawl" directives. This involves using Workers AI with open source models to pre-generate HTML pages, storing them in R2, and integrating them as hidden links via custom HTML transformation. AI Labyrinth also acts as a next-generation honeypot, identifying and fingerprinting bad bots by analyzing which crawlers follow these hidden pathways, feeding data into machine learning models for improved bot detection.

Mar 2025Global expansion in Generative AI: a year of growth, newcomers, and attacksThis post analyzes the growth and traffic patterns of Generative AI services throughout 2024 and early 2025, using aggregated data from Cloudflare's network and DNS resolver. It highlights key trends such as ChatGPT's continued popularity, rapid traffic growth across Generative AI services (251% increase), the rise of new entrants like DeepSeek and Grok/xAI, global reach with regional variations in user distribution, and a significant volume of targeted cyberattacks (197 billion blocked requests). It provides specific rankings of popular Generative AI services and their visitor location distributions.

Feb 2025Keep AI interactions secure and risk-free with Guardrails in AI GatewayThis post introduces Guardrails in AI Gateway, a new feature that integrates Llama Guard 3 8B, hosted on Workers AI, for content moderation of AI interactions. Guardrails allows developers to configure evaluation scope (prompts, responses, or both) and specify hazard categories to block or flag. The implementation leverages the AI Binding for seamless integration with Workers and Workers AI, and includes eventID for traceability. The post details how Guardrails prevents harmful prompts from reaching models and inspects responses, providing examples of blocked prompts and the underlying Llama Guard evaluation.

Feb 2025Making Cloudflare the best platform for building AI AgentsThis post introduces the `agents-sdk` framework for building AI agents on Cloudflare Workers, enabling features like real-time communication, state persistence via Durable Objects, and integration with React applications. It also announces updates to Workers AI, including JSON mode support, longer context windows, and improved tool calling capabilities, which are crucial for agent functionality. The `AIChatAgent` class and agent routing are highlighted as key components for developing and deploying agents at scale.

Feb 2025No hallucinations here: track the latest AI trends with expanded insights on Cloudflare RadarThis post introduces the "AI Insights" page on Cloudflare Radar, which expands visibility into AI trends. It incorporates the existing AI bot & crawler traffic graph and adds new metrics: relative popularity of Generative AI services (based on 1.1.1.1 DNS traffic), analysis of robots.txt files for AI bot directives across top domains, and popularity of models and tasks on Workers AI (based on customer account share). The underlying data for these new metrics is made available via the Radar API and Data Explorer.

Feb 2025Preserving content provenance by integrating Content Credentials into Cloudflare ImagesThis post introduces the integration of the Coalition for Content Provenance and Authenticity (C2PA) standard into Cloudflare Images. This feature allows content creators and publishers to preserve the provenance chain of images, including creation details and subsequent edits, across the Cloudflare network. When images are transformed by Cloudflare Images (e.g., resizing, format changes), the existing Content Credentials are preserved, and Cloudflare cryptographically signs any additional transformations. This ensures that end-users can verify the full provenance chain using open-source verification services. The implementation involves embedding Content Credentials using JUMBF and cryptographically signing assertions with Cloudflare's end-entity certificate, which is part of a Trust List.

Jan 2025Demonstrating reduction of vulnerability classes: a key step in CISA’s “Secure by Design” pledgeThis post details Cloudflare's proactive approach to reducing entire classes of vulnerabilities, specifically injection and secrets, aligning with CISA's 'Secure by Design' pledge. It highlights the implementation of custom rulesets, 'build break' enforcement in CI/CD pipelines, and developer enablement to prevent vulnerabilities at the source, leading to significant reductions in secrets (79%) and potential SQL/code injection vulnerabilities (44%). This demonstrates a concrete application of secure-by-design principles to enhance product security.

Dec 2024Un experimento rápido: translating Cloudflare Stream captions with Workers AIThis post details an experiment using Cloudflare Workers AI and the m2m100-1.2b translation model to translate automatically generated VTT captions for Cloudflare Stream videos. It describes the process of parsing VTT files, running translation inference calls in parallel using `Promise.all`, and re-exporting the translated captions back to VTT format. The post highlights the impact of fragmented input text on translation quality and demonstrates an improvement by implementing a pre-processing step to consolidate caption cues into full sentences before translation. It also notes the rapid development cycle possible with Workers AI and the need for language expertise to evaluate translation quality.

Dec 2024Hi Claude, build an MCP server on Cloudflare WorkersThis post introduces the `workers-mcp` tooling, a new library that simplifies the creation of MCP servers on Cloudflare Workers. It demonstrates how to define tools using JSDoc annotations and integrate with Workers AI for image generation. The post contrasts this simplified approach with the more verbose manual MCP server implementation, highlighting the benefits of Cloudflare's integrated platform for AI agent development.

Dec 2024Bring multimodal real-time interaction to your AI applications with Cloudflare CallsThis post details the integration of Cloudflare Calls with OpenAI's Realtime API to enable multimodal, real-time interaction with AI applications. It explains how Cloudflare Calls leverages WebRTC to overcome the latency limitations of WebSockets for audio and video streams, allowing multiple users to interact with AI simultaneously. The post provides a code example demonstrating how to connect human and AI sessions by adding AI audio tracks to human sessions and vice-versa, facilitating bidirectional communication.

Dec 2024Robotcop: enforcing your robots.txt policies and stopping bots before they reach your websiteIntroduces Robotcop, an extension of AI Crawl Control, that allows customers to enforce their robots.txt policies for AI bots at the network level via WAF rules. This moves beyond voluntary compliance to programmatic enforcement of content access rules for AI crawlers.

Nov 2024DO it again: how we used Durable Objects to add WebSockets support and authentication to AI GatewayThis post details the implementation of WebSocket support and authentication for Cloudflare's AI Gateway. It describes how Durable Objects are used to manage persistent WebSocket connections, handle authentication via API tokens (including a fallback for browser limitations using the 'sec-websocket-protocol' header), and route requests to various AI providers. The post also introduces the `eventId` field for correlating asynchronous messages in streaming scenarios and provides code examples for both Node.js and browser WebSocket clients.

Oct 2024Billions and billions (of logs): scaling AI Gateway with the Cloudflare Developer PlatformThis post details the technical evolution of Cloudflare AI Gateway's log storage, migrating from a 30-minute retention period using D1 and a backend worker to indefinite storage. The solution involves storing request bodies in R2 and using Durable Objects (DOs) for log indexing and transactional storage. To overcome DO limitations (10 million logs per DO), logs are sharded by account ID and gateway name, raising the effective limit to 100 million logs per account. An Account Manager DO was introduced to track user entitlements, gateway limits, and log storage capacity, ensuring fair usage and system integrity. Gateways update the Account Manager periodically after every 1000 inference requests.

Oct 2024Building Vectorize, a distributed vector database, on Cloudflare’s Developer PlatformThis post details the technical architecture and implementation of Vectorize, a new globally distributed vector database built on Cloudflare's Developer Platform. It explains the core concepts of vector databases, the challenges of high-dimensional data, and the specific techniques used in Vectorize, including Inverted File Index (IVF) for pruning the search space, Product Quantization (PQ) for vector compression, and a result refinement pass for accuracy. It also describes the eventual consistency model and the use of R2 and Durable Objects for storing and managing index data and the write-ahead log.

Sep 2024AI Everywhere with the WAF Rule Builder Assistant, Cloudflare Radar AI Insights, and updated AI bot protectionThis post introduces the AI Assistant for WAF Rule Builder, allowing users to generate WAF rules using natural language prompts. It also adds AI bot traffic insights to Cloudflare Radar, providing visibility into the activity of AI bots and crawlers. Furthermore, it enhances AI bot protection by expanding the 'block AI bots' feature to include more AI bot signatures and enabling dynamic updates for the AI bot rule, moving towards a managed rule framework. The post also details the impact of these new fingerprints on Cloudflare's ML model and observes a decline in AI bot traffic, hypothesizing that Cloudflare's blocking features are discouraging them.

Sep 2024Network trends and natural language: Cloudflare Radar’s new Data Explorer & AI AssistantThis post details the development of an AI Assistant for Cloudflare Radar's Data Explorer. The assistant uses Cloudflare Workers AI to translate natural language questions into Radar API calls. The implementation involves a multi-stage LLM inference process: first, determining the topic of the user's question, and then generating specific API endpoint descriptions relevant to that topic. This approach avoids fine-tuning and allows for flexibility in model selection and parameter combination. The post highlights the use of prompt engineering and the challenges of handling a large number of API endpoints and parameters, demonstrating a practical application of LLMs for enhancing data accessibility and user experience.

Sep 2024Cloudflare’s bigger, better, faster AI platformThis post details significant upgrades to Cloudflare's AI platform, focusing on Workers AI, AI Gateway, and Vectorize. For Workers AI, it announces the addition of more powerful GPUs enabling faster inference and support for larger models (e.g., Llama 3.1 70B), expansion of GPU presence to over 180 cities, and a new unit-based pricing model replacing 'neurons'. It also introduces the 'Run Any* Model' feature in closed beta for greater model flexibility. For AI Gateway, new features include persistent logs (open beta) with increased storage capacity, enhanced logging interface with cost/duration insights and human feedback annotations, custom metadata tagging, and Logpush export. AI evaluations with human-in-the-loop feedback are also in open beta. Vectorize is announced as generally available with expanded index sizes and faster queries. The post also references a technical blog post detailing the optimizations for Workers AI speed, including KV cache compression, speculative decoding, and new hardware design.

Sep 2024Making Workers AI faster and more efficient: Performance optimization with KV cache compression and speculative decodingThis post details performance optimizations for Cloudflare's Workers AI platform. It introduces upgraded hardware (12th generation compute servers) enabling higher throughput for Llama 3.1 and 3.2 models. It also describes a novel, open-source KV cache compression technique using PagedAttention, which allows for up to 64x compression and a 5.18x increase in throughput by efficiently managing memory allocation for attention heads and sequence lengths. Additionally, speculative decoding, specifically prompt-lookup decoding, is implemented to increase generation speed by up to 70% for Llama 3.1 70B models, albeit with a potential trade-off in output quality.

Sep 2024Start auditing and controlling the AI models accessing your contentThis post introduces new tools for site owners to audit and control how AI models access their content. It includes an analytics view of AI services crawling sites, a one-click option to block all AI crawlers, and filters in the WAF to allow specific AI providers or bot types. It also previews a monetization feature for content usage by AI models.

Sep 2024Protecting APIs from abuse using sequence learning and variable order Markov chainsThis post details how Cloudflare is using statistical machine learning techniques, specifically sequence learning with variable order Markov chains, to protect APIs against sequential abuse. It introduces the concept of API sessions and explains how these models help distinguish malicious from benign API request sequences, building upon previous work in API sequence analytics.

Jul 2024Making WAF ML models go brrr: saving decades of processing timeThis post details performance optimizations for Cloudflare's WAF ML models, achieving a 5.5x speedup (82% reduction in execution time) by optimizing the feature extraction process. The primary optimization involved replacing `std::collections::HashMap` lookups with the Aho-Corasick algorithm (specifically using `AhoCorasickKind::DFA`) for ngram processing, reducing lookup time and improving cache efficiency. Additionally, the post mentions optimizations for `regex::regex::bytes::Regex::replace_all` and discusses the overall system design of the WAF Attack Score, including normalization, feature extraction, and model inference.

Jun 2024Embedded function calling in Workers AI: easier, smarter, fasterIntroduces embedded function calling in Workers AI, where LLM inference and function execution occur in the same runtime, eliminating network round-trips. This is facilitated by the new `ai-utils` npm package, which includes the `runWithTools` method for executing functions and handling complex agentic chains, and `createToolsFromOpenAPISpec` for dynamically generating tools from OpenAPI specifications. The `autoTrimTools` helper function is also introduced to optimize token usage by intelligently selecting relevant tools.

Jun 2024Using machine learning to detect bot attacks that leverage residential proxiesThis post details the development and implementation of Cloudflare's Bot Management machine learning model v8, which specifically targets and detects bot attacks utilizing residential proxies. It explains the challenges posed by residential proxies, the evolution of Cloudflare's bot detection models since 2019, the training methodology for v8 including data sourcing and validation, and the specific ML features used. The post highlights how v8 analyzes network and behavioral signals to identify residential proxy abuse without relying on IP blocking, thus reducing false positives.

Jun 2024Introducing Stream Generated Captions, powered by Workers AIThis post details the integration of the Whisper model via Cloudflare Workers AI to provide automatic caption generation for Cloudflare Stream. It describes the pre-processing steps to format audio into 30-second batches compatible with Whisper, the parallelization of requests to Workers AI, and the re-ordering of out-of-order responses to ensure sequential caption alignment. The technical implementation uses a simple Worker script to call the `ai.run` function with the Whisper model.

May 2024AI Gateway is generally available: a unified interface for managing and scaling your generative AI workloadsAI Gateway is now generally available, acting as a proxy for generative AI workloads. It provides a unified interface for managing and scaling AI ops, offering features like analytics (request, token, cost tracking), real-time logs, caching rules, rate limiting, support for Workers AI and 10+ external providers (including Groq and Cohere), and universal endpoints with request fallbacks. The post details a code example for integrating with OpenAI and a curl example for using multiple providers. Future plans include persistent logs, custom metadata, prompt management, and fine-tuning capabilities.

Apr 2024Meta Llama 3 available on Cloudflare Workers AIThis post announces the availability of Meta Llama 3 8B on Cloudflare Workers AI. It details the model's performance, tokenizer (tiktoken-based, 128k vocabulary), and architecture (grouped-query attention). It also provides a code example for running inference with Llama 3 via the Workers AI platform and mentions upcoming releases of other Meta AI tools.

Apr 2024Leveling up Workers AI: general availability and more new capabilitiesThis post announces the General Availability of Workers AI, detailing improvements in load balancing for greater reliability and performance, with GPUs deployed in over 150 cities. It highlights significant cost reductions for popular models like Llama 2 and Mistral 7B, and introduces new pricing structures including a free tier of 10,000 neurons per day. New dashboard and playground tools are also introduced. The post announces an expansion of the Hugging Face partnership, adding four new models (Mistral 7B v0.2, Nous Research's Hermes 2 Pro, Google's Gemma 7B, and Starling-LM-7B-beta) served via Hugging Face's TGI backend. A key new capability is the support for Bring Your Own LoRAs (BYO LoRAs) for fine-tuned inference, currently in open beta for specific models and adapter sizes. Additionally, Python support for Cloudflare Workers is launched in open beta, allowing developers to write Workers in Python using Pyodide and integrate with various Cloudflare bindings and frameworks like LangChain.

Apr 2024Running fine-tuned models on Workers AI with LoRAsThis post introduces the open beta of Bring Your Own (BYO) LoRAs on Workers AI, enabling users to run fine-tuned inference with LoRA adapters for Mistral, Gemma, and Llama 2 models. It explains the concepts of fine-tuning and LoRA, detailing the mathematical principles behind LoRA's efficiency. The post outlines how to use LoRAs with Workers AI, including supported base models and current limitations (e.g., no quantized models, adapter size limits). It also delves into the technical implementation of multi-tenant LoRA serving, highlighting the use of the Punica CUDA kernel design and global cache optimizations to achieve low latency and high throughput.

Mar 2024Mitigating a token-length side-channel attack in our AI productsImplemented a mitigation for a token-length side-channel attack affecting streaming AI responses by adding a 'p' property with a variable-length random string to JSON objects in streaming responses. This obfuscates the actual token length, preventing attackers from inferring information from network packet lengths. This mitigation is applied automatically to both Workers AI and AI Gateway.

Mar 2024Cloudflare announces Firewall for AIIntroduces Firewall for AI, a new security layer designed to protect Large Language Models (LLMs) from various attacks including prompt injection, model denial of service, and sensitive information disclosure. It leverages existing WAF capabilities like Rate Limiting and Sensitive Data Detection, and adds new prompt validation to identify and block malicious inputs before they reach LLMs. The post details the unique attack surfaces introduced by LLMs compared to traditional applications, referencing the OWASP Top 10 for LLMs.

Mar 2024Dispelling the Generative AI fear: how Cloudflare secures inboxes against AI-enhanced phishingThis post details how Cloudflare's existing machine learning systems, trained on billions of emails, are leveraged to defend against AI-enhanced phishing attacks, including Business Email Compromise (BEC). It explains how various signals such as sender reputation, correspondence patterns, metadata, sentiment analysis, domain mismatches, and social graph analysis are used to detect malicious emails. The post also discusses the inherent limitations and trade-offs of AI-generated attacks, emphasizing that opportunity and access remain critical factors for attackers. Cloudflare's multi-layered security approach is highlighted as a robust defense against these evolving threats.

Mar 2024Cloudflare launches AI Assistant for Security AnalyticsIntroduces an AI assistant for Cloudflare's Security Analytics dashboard. This feature leverages the Workers AI platform and off-the-shelf LLMs to convert natural language queries into GraphQL filters for security event data. The system generates time series charts based on these queries, enabling users to more quickly discover anomalies and potential security attacks without manual filter configuration. It emphasizes privacy by keeping customer data within Cloudflare's network and not using it for model training.

Mar 2024Defensive AI: Cloudflare’s framework for defending against next-gen threatsThis post introduces Cloudflare's 'Defensive AI' framework, detailing its application in enhancing security products. It highlights the development of API Anomaly Detection for API Gateway, which uses ML to model expected API request sequences and identify deviations. For the WAF, it discusses the evolution of WAF Attack Score, an ML model trained on attack traffic to detect new and undiscovered attacks. In email security, it describes Honeycomb (sender domain reputation) and Labyrinth (per-customer spoof detection) as AI-powered protections against phishing. Finally, it introduces Zero Trust user risk scoring, which uses AI to analyze user behavior anomalies for enhanced access control.

Feb 2024Unlocking new use cases with 17 new models in Workers AI, including new LLMs, image generation models, and moreThis post announces the addition of 17 new models to the Workers AI catalog, expanding its capabilities. New text generation models include @cf/deepseek-ai/deepseek-math-7b-instruct, @cf/openchat/openchat-3.5-0106, @cf/microsoft/phi-2, @cf/tinyllama/tinyllama-1.1b-chat-v1.0, @cf/thebloke/discolm-german-7b-v1-awq, @cf/qwen/qwen1.5-0.5b-chat, @cf/qwen/qwen1.5-1.8b-chat, @cf/qwen/qwen1.5-7b-chat-awq, @cf/qwen/qwen1.5-14b-chat-awq, @cf/tiiuae/falcon-7b-instruct, and @cf/defog/sqlcoder-7b-2. A summarization model, @cf/facebook/bart-large-cnn, is also added. For text-to-image, new models include @cf/lykon/dreamshaper-8-lcm, @cf/runwayml/stable-diffusion-v1-5-inpainting, @cf/runwayml/stable-diffusion-v1-5-img2img, and @cf/bytedance/stable-diffusion-xl-lightning. An image-to-text model, @cf/unum/uform-gen2-qwen-500m, is also available. The post highlights new language models with improved multilingual support and smaller, faster SLMs. It also details new Stable Diffusion pipelines for image-to-image and inpainting use cases, including code examples. The model catalog is being optimized using ONNX and TensorRT formats.

Feb 2024Monitoring machine learning models for bot detectionThis post details Cloudflare's approach to monitoring machine learning models used for bot detection. It explains the importance of monitoring for maintaining model performance in production, especially against evolving threats. The post outlines the process of using ground-truth datasets, specializations for granular analysis, and the integration of monitoring into the Endeavor ML platform. It also describes the technical implementation of rendering monitoring queries, computing performance metrics, storing metrics, and running tasks on GPUs.

Feb 2024Adding new LLMs, text classification and code generation models to the Workers AI catalogThis post announces the addition of eight new models to the Workers AI catalog: six text generation models (@hf/thebloke/llama-2-13b-chat-awq, @hf/thebloke/zephyr-7b-beta-awq, @hf/thebloke/mistral-7b-instruct-v0.1-awq, @hf/thebloke/openhermes-2.5-mistral-7b-awq, @hf/thebloke/neural-chat-7b-v3-1-awq, @hf/thebloke/llamaguard-7b-awq) and two code generation models (@hf/thebloke/deepseek-coder-6.7b-base-awq, @hf/thebloke/deepseek-coder-6.7b-instruct-awq). It highlights the performance advantages of DeepSeek Coder and OpenHermes-2.5-Mistral-7B, explains Activation-aware Weight Quantization (AWQ) for memory efficiency, and introduces LlamaGuard for prompt and response safety classification with a code example. The post emphasizes the ease of use of Workers AI for developers.

Jan 2024LangChain Support for Workers AI, Vectorize and D1This post details the integration of LangChain with five Cloudflare AI tools: Workers AI Chat Models, Workers AI Instruct Models, Workers AI Text Embeddings Models, Vectorize Vector Store, and Cloudflare D1-Backed Chat Memory. It provides examples and a template for building AI-powered applications using these components, including a retrieval augmented generation (RAG) chatbot that routes between multiple data sources.

Dec 2023A look inside the Cloudflare ML Ops platformThis post details Cloudflare's internal ML Ops platform for model training. Key contributions include: 1. Notebooks: Deployment of JupyterHub on Kubernetes for scalable and collaborative data exploration and model experimentation. 2. GitOps: Adoption of GitOps with ArgoCD for continuous delivery and infrastructure management, enabling declarative infrastructure and application code management via pull requests. 3. Templates: Introduction of production-ready model templates for training (ETL, feature engineering, experiment tracking, DAG orchestration), batch inference, stream inference (FastAPI microservices), and explainability (Streamlit/Bokeh dashboards). 4. Orchestration: Support for multiple open-source orchestration tools including Apache Airflow, Argo Workflows, Kubeflow Pipelines, and Temporal, allowing teams to choose based on use case and preference. 5. Hardware: Focus on optimizing GPU and CPU utilization for core datacenter workloads and edge inference.

Nov 2023Workers AI Update: Stable Diffusion, Code Llama + Workers AI in 100 citiesThis post announces the availability of Stable Diffusion XL 1.0 and Code Llama models within Workers AI. It details how to use these models via API calls and Cloudflare Workers, providing code examples for both image generation and code generation. A significant expansion of Workers AI inference to over 100 cities globally is also highlighted, aiming to reduce inference latency by running models closer to users. Mistral 7B is also mentioned as a recent addition.

Nov 2023Workers AI Update: Hello, Mistral 7B!This post introduces the Mistral 7B model to Cloudflare Workers AI, highlighting its performance advantages due to grouped-query attention. It provides technical details on how to use the model via REST API and Worker scripts, and explains the concepts of attention, multi-head attention, multi-query attention, and grouped-query attention with diagrams and references to relevant research papers. The post also discusses the trade-offs between these attention mechanisms in terms of quality, memory usage, and inference speed.

Nov 2023Streaming and longer context lengths for LLMs on Workers AIThis post introduces streaming response capabilities for all LLMs on Workers AI, enabling token-by-token delivery of responses using Server-Sent Events (SSE). It also announces increased context and sequence lengths for the Llama-2 model (from 768/256 to 2048/1800 for int8, and 3072/2500 for fp16) and the addition of a 16-bit full-precision Llama-2 variant. Examples are provided for using SSE with the REST API and a Worker script, along with client-side JavaScript for consuming the event stream.

Sep 2023Privacy-preserving measurement and machine learningThis post details Cloudflare's participation in the IETF's Privacy Preserving Measurement working group and their implementation of the Distributed Aggregation Protocol (DAP) in their open-source aggregator server, Daphne. It explains the core principles of DAP, including data minimization and the use of secret sharing and multi-party computation to aggregate data without exposing individual measurements. The post also discusses challenges like input validation and non-linear aggregation, and how zero-knowledge proofs and Prio are used to address them.

Sep 2023Easily manage AI crawlers with our new bot categoriesThis post introduces new bot categories within Cloudflare's WAF and Rules Engine, specifically enabling users to easily allow or block AI crawlers. It highlights the existing protection against malicious crawlers and expands on the Verified Bot directory by adding AI Crawler as a distinct category. The post also outlines criteria for 'respectful AI bots' and discusses the development of industry-wide protocols for AI crawler management, demonstrating a proactive stance on integrating AI responsibly.

Sep 2023Cloudflare Integrations Marketplace introduces three new partners: Sentry, Momento and TursoThis post introduces three new integrations to the Cloudflare Integrations Marketplace for Workers: Sentry for application monitoring (errors, exceptions, logs), Turso for edge-hosted distributed databases (libSQL/SQLite), and Momento for serverless caching. The Sentry integration uses a Tail Worker to capture and send data, with future plans for source map uploads and deployment correlation. The Turso integration leverages its global replicas and Smart Placement for read-heavy workloads. The Momento integration uses an OAuth2 flow to add API keys as secrets for SDK usage.

Sep 2023The best place on Region: Earth for inferenceThis post announces Cloudflare's entry into the AI space with three new products: Workers AI (serverless GPU inference), Vectorize (vector database), and AI Gateway (caching, rate limiting, and observability for AI deployments). It details the rationale for placing inference workloads on the network edge due to latency sensitivity. It also highlights partnerships with NVIDIA, Microsoft, Hugging Face, Databricks, and Meta, and discusses the role of ONNX runtime for model portability across devices, edge, and cloud.

Sep 2023Workers AI: serverless GPU-powered inference on Cloudflare’s global networkIntroduced Workers AI, a serverless platform for running AI inference on Cloudflare's global GPU network. It offers a curated set of open-source models (LLMs, ASR, translation, classification, image classification, embeddings) accessible via Workers, Pages Functions, or a REST API. The platform emphasizes developer experience, privacy by default (no model training on user data), and pay-as-you-go pricing. It integrates with Vectorize for building context-aware AI applications.

Sep 2023Writing poems using LLama 2 on Workers AIThis post demonstrates the practical application of Workers AI by showcasing a 14-line JavaScript code snippet that utilizes the Llama 2 model to generate poems. It provides example prompts and their corresponding outputs, illustrating the model's ability to adapt to different styles and themes. The post highlights the ease of deployment and use of Workers AI for generative AI tasks.

Sep 2023Announcing AI Gateway: making AI applications more observable, reliable, and scalableThis post introduces AI Gateway, a new product that acts as a portal for making AI applications more observable, reliable, and scalable. It sits between applications and AI APIs (like OpenAI), offering features such as response caching, request limiting and retries, and analytics for monitoring usage. It supports dedicated endpoints for providers like OpenAI, Hugging Face, and Replicate, as well as a universal endpoint for more flexibility with fallback models and retries. AI Gateway provides analytics on request count, users, cost, and duration, and aggregates data across multiple providers. It also offers caching and rate limiting to control costs and scale applications sustainably. AI Gateway is designed to integrate with Workers AI and Vectorize, forming a comprehensive Workers platform for AI applications. Future plans include usage alerts, jailbreak protection, dynamic model routing with A/B testing, and advanced cache rules. An enterprise use case is also described, where AI Gateway can monitor and observe AI usage within organizations, log requests, apply access policies, and implement data loss prevention (DLP) strategies.

Sep 2023Partnering with Hugging Face to make deploying AI easier and more affordable than ever 🤗This post announces a partnership with Hugging Face to enhance AI model deployment. Key contributions include bringing serverless GPU models to Hugging Face, integrating Hugging Face models into Cloudflare's catalog, and enabling Cloudflare integrations within Hugging Face's Inference solutions. This expands Cloudflare's AI capabilities by making it easier for developers to deploy and access a wider range of AI models.

Sep 2023Vectorize: a vector database for shipping AI-powered applications to production, fastIntroduced Vectorize, a new vector database service available on Cloudflare Workers. Vectorize enables developers to build AI-powered applications by storing and querying vector embeddings, facilitating use cases like semantic search, classification, recommendation, and anomaly detection. It integrates with Workers AI and supports bringing your own embeddings from platforms like OpenAI and Cohere. The post details the necessity of vector databases for ML models to retain context and provides a code example demonstrating vector insertion and querying for a recommendation engine.

Sep 2023What AI companies are building with CloudflareThis post details how multiple AI companies (Azule.ai, 42able.ai, Layercode, Eclipse AI, Embley, Chainfuse) are leveraging Cloudflare's Developer Platform to build and scale their AI applications. It highlights the use of Workers, Pages, R2, KV, D1, Workers AI (Constellation), Durable Objects, WebRTC, and API Gateway for various aspects of AI development, including model hosting, data storage (including vector storage), real-time chat, API backends, and frontend hosting. Specific examples include Azule.ai using R2 for on-disk vector search and Eclipse AI achieving a 70% cost reduction by switching from S3 to R2 for object storage.

Sep 2023Cloudflare’s 2023 Annual Founders’ LetterThis post announces the expansion of Cloudflare's global network to support AI inference by deploying GPU resources in over 100 cities by the end of 2023 and nearly every city by the end of 2024. It highlights the benefits of running AI inference at the edge, contrasting it with on-device and centralized public cloud solutions. The post also mentions the use of Meta's Llama 2 LLM running on Cloudflare's Workers AI platform for generating a poem.

Jul 2023Cloudflare Zaraz steps up: general availability and new pricingThis post announces the general availability of Cloudflare Zaraz and introduces its new pricing structure. Zaraz is a solution for managing third-party tools (like Google Analytics, Facebook CAPI) by transitioning to server-side data collection. It is now part of the Cloudflare Developer Platform and included in free, paid, and enterprise plans. Pricing is based on 'Zaraz Loads' (page loads activating the script or Pageview trigger) and feature sets. The free plan includes 100,000 loads/month, with paid plans offering more loads and advanced features like E-commerce, Custom Endpoints, Workers Variables, and Privacy Features. The post also mentions the ongoing reevaluation of pricing based on customer feedback.

Jun 2023Cloudflare Zaraz supports JSONataThis post introduces JSONata integration into Cloudflare Zaraz, allowing users to transform data payloads sent to third-party tools using a JSON query and transformation language. This provides a more flexible data manipulation capability than simple field mapping, without the overhead of a full Cloudflare Worker. Examples demonstrate using JSONata for string manipulation (e.g., converting to lowercase) and calculations (e.g., summing product prices and quantities). The Zaraz Context object, containing event data and automatically gathered information, is accessible to JSONata expressions.

Jun 2023Globally distributed AI and a Constellation updateThis post announces updates to Cloudflare's Constellation platform, which enables globally distributed AI inference. Key updates include increasing the model size limit from 10MB to 50MB, introducing tensor caching to reduce network and parsing overhead for repeated inference tasks with similar inputs, and adding support for the XGBoost runtime in addition to the existing ONNX runtime. The post also discusses the broader implications of globally distributed AI for custom web experiences and making devices smarter, highlighting the performance benefits of running inference closer to users.

Jun 2023Every request, every microsecond: scalable machine learning at CloudflareThis post details the evolution of Cloudflare's machine learning feature serving infrastructure. It describes the transition from a Unix Domain Socket (UDS) based system (Gagarin) to using memory-mapped files for inter-process communication (IPC). The challenges with UDS, including high tail latency, suboptimal resource utilization, and scalability constraints, are highlighted. The new approach leverages memory-mapped files for reduced latency and CPU contention, combined with wait-free synchronization and efficient data deserialization techniques to serve millions of ML features per request.

Jun 2023How Cloudflare runs machine learning inference in microsecondsThis post details optimizations applied to the machine-learning models within Cloudflare's bot management module, reducing P50 latency by 20% (from 388μs to 309μs) and P99 latency by 14% (from 940μs to 813μs). Key optimizations include: 1. Engineering for zero allocations by using stack allocation for fixed-size buffers and re-using pre-allocated buffers. 2. Choosing algorithms that operate in-place and store state on the stack, such as using iterators for case-insensitive string comparison. 3. Implementing automated testing for memory allocations using `dhat`. 4. Optimizing CatBoost for single-document evaluation by passing references to contiguous memory instead of allocating vectors. 5. Rewriting Rust bindings to accept slices (`&[f32]`, `&[&str]`) instead of owned `Vec`s and pre-computing hashes for categorical features to enable buffer reuse across multiple model executions.

May 2023Announcing Cloudflare Secrets StoreIntroduces Cloudflare Secrets Store, a new secrets management offering. This expands on previous capabilities like environment variables and secrets for Cloudflare Workers by providing account-level secrets that can be referenced across multiple Workers and other Cloudflare services like WAF. It emphasizes security features such as scoped permissions, audit logs, and on-demand encryption key rotation. The post highlights the evolution from narrowly scoped Worker secrets to a centralized solution for managing sensitive data across the Cloudflare ecosystem.

May 2023Using LangChainJS and Cloudflare Workers togetherThis post details the integration of LangChainJS with Cloudflare Workers, enabling developers to build AI-powered applications at the edge. It provides a step-by-step guide on setting up a Cloudflare Worker, installing LangChainJS, and using document loaders (specifically CheerioWebBaseLoader) to fetch and process web content. The post also introduces the concepts of text splitting for managing LLM input size and the role of embeddings and vector stores in understanding text relevance for LLM interactions.

May 2023Cloudflare R2 and MosaicML enable training LLMs on any compute, anywhere in the world, with zero switching costsThis post details the integration of Cloudflare R2 object storage with MosaicML's StreamingDataset and Composer libraries to enable efficient training of large language models (LLMs) and diffusion models. It highlights how R2's zero-egress pricing eliminates vendor lock-in and allows for flexible compute provider selection. The post provides code examples for writing data to R2 using MDSWriter, uploading to R2 via awscli, and reading data into PyTorch DataLoaders with StreamingDataset. It also demonstrates saving and loading model checkpoints to R2 using Composer, emphasizing the elimination of expensive shared file systems. The integration allows for seamless training job orchestration across multiple cloud providers.

May 2023Introducing Constellation, bringing AI to the Cloudflare stackIntroduced Constellation, a new capability within the Cloudflare stack that enables developers to run pre-trained machine learning models and inference tasks on Cloudflare's network. Constellation allows for fast, low-latency inference directly within Cloudflare Workers scripts, supporting various ML tasks such as image classification, object detection, NLP, and more. Developers can upload supported models (e.g., ONNX format) or use a catalog of verified models. The post provides examples of using Constellation for image classification with the SqueezeNet model and for email sentiment analysis using the t5-small model, demonstrating its integration with Workers and other Cloudflare services like Email Routing.

May 2023A complete suite of Zero Trust security tools to help get the most from AIThis post details how Cloudflare One's Zero Trust security tools can be applied to manage the risks associated with using AI services. It introduces features for measuring AI usage (observing SaaS application traffic), controlling API access to sensitive data using service tokens and Cloudflare Tunnel, restricting data uploads with Data Loss Prevention (DLP) policies, and scanning SaaS applications for misconfigurations using Cloud Access Security Broker (CASB).

May 2023Introducing Cursor: the Cloudflare AI AssistantIntroduces Cursor, an experimental AI assistant for Cloudflare's Developer Platform, built using Workers, Durable Objects, OpenAI, and Cloudflare developer docs. It employs a "Search-Ask" method to answer questions about the Developer Platform and provide links to relevant documentation. This represents a first step towards AI-driven developer experience and is labeled experimental due to its early stage.

May 2023Query Cloudflare Radar and our docs using ChatGPT pluginsThis post details the development of two ChatGPT plugins: Cloudflare Radar and Cloudflare Docs. The Radar plugin leverages Cloudflare Radar's public API and an OpenAPI schema generator (itty-router-openapi) to provide real-time internet data insights to ChatGPT. The Docs plugin uses Cloudflare Workers and KV as a vector store to provide up-to-date documentation snippets to ChatGPT, with a scheduler Worker for refreshing the index. It demonstrates the use of Workers for both front-facing APIs and scheduled batch processing.

May 2023Batteries included: how AI will transform the who and how of programmingThis post introduces AI assistants built into the Cloudflare Workers platform to enhance programmer productivity and broaden access to the Cloudflare Developer Platform. It also discusses the architectural implications of AI systems requiring significant data movement and how Cloudflare's platform, including R2, facilitates data fluidity.

Mar 2023Using the power of Cloudflare’s global network to detect malicious domains using machine learningThis post details the application of machine learning, specifically transformer-based neural networks, to detect malicious domains. It describes the process of training a model using DNS query data and known DGA domains, achieving over 99% accuracy. The post also explains the detection of DNS tunneling by analyzing query patterns and distinguishing them from benign applications. The deployment strategy involves processing DNS query names from Cloudflare's 1.1.1.1 resolver to identify and classify DGA domains as soon as they are observed.

Oct 2022Stronger than a promise: proving Oblivious HTTP privacy propertiesThis post details the formal, computer-aided security analysis of the Oblivious HTTP (OHTTP) protocol, a new privacy-preserving protocol for decoupling 'who' from 'what' was sent. It describes the protocol's design, including its use of HPKE and Binary HTTP, and presents a simplified security model and goals. The analysis uses the Tamarin prover to model the protocol's behavior and prove its privacy properties against an attacker who can compromise either the relay or the gateway, but not both.

Sep 2022Cloudflare Zaraz supports Managed Components and DLP to make third-party tools privateThis post introduces Cloudflare Zaraz's new Managed Components and Data Loss Prevention (DLP) features. Managed Components offer a more secure and privacy-aware way to load third-party tools by running them in an isolated environment within a Component Manager. This allows for granular control over permissions and the data exposed to these components. The DLP features further enhance privacy by enabling the scanning, reporting, masking, and removal of PII from information shared with third-party tools, based on content rather than just field type. This provides a robust mechanism for fencing information before it is accessed by components.

Mar 2022Evolving our machine learning to stop mobile botsThis post details the evolution of Cloudflare's Bot Management machine learning models, specifically focusing on adapting to the increasing volume of mobile traffic. It describes the iterative process of model building, data gathering (including new techniques for labeling mobile app traffic), feature selection using ANOVA f-value, model training with Catboost, and evaluation using SHAP Explainer. The deployment process includes shadow mode and phased rollouts. The key technical contribution is the development of new datasets and training methodologies to improve the accuracy of bot detection for native mobile applications, resulting in significantly lower false positive rates.

Jan 2022Landscape of API TrafficThis post significantly expands the understanding of Cloudflare's evolving capabilities by highlighting the increasing importance of APIs in internet traffic. It details how Cloudflare is leveraging its global network data to analyze API trends, identify key industries driving this growth (like Banking and Financial Services), and understand API characteristics (methods, payload encoding). Crucially, it introduces Cloudflare's API Shield product suite (API Discovery, Schema Validation, mTLS, API Abuse Detection) as a direct response to the challenges posed by API traffic, demonstrating a proactive approach to securing this critical communication layer. The post also notes the adoption of newer protocols like TLS v1.3, QUIC, and gRPC for APIs, indicating an evolution in the underlying technologies Cloudflare supports and secures.

Oct 2021Get started Building Web3 Apps with CloudflareThis post introduces an open-source, full-stack application demonstrating how to mint and manage NFTs using Cloudflare Workers and Cloudflare Pages. It details the architecture, including a smart contract deployed on Ethereum's Rinkeby network, a web interface for claiming NFTs, and Cloudflare Workers for generating JSON metadata. It also highlights the use of Infura for pinning and Cloudflare IPFS Gateway for serving NFT images, showcasing how Cloudflare's developer platform can scale Web3 applications.

Apr 2021Bringing AI to the edge with NVIDIA GPUsThis post announces the integration of NVIDIA GPUs and TensorFlow support into the Cloudflare Workers platform, enabling AI model inference at the edge. It details the partnership with NVIDIA, the benefits of edge AI (low latency, scalability, security), and the use of TensorFlow for model development. A demonstration website, nataornot.com, is presented as an example of an AI-based application running on this infrastructure.

Oct 2020Exploring WebAssembly AI Services on Cloudflare WorkersThis post details an exploration of deploying AI services using WebAssembly (WASM) on Cloudflare Workers, in collaboration with Latent AI. It highlights the benefits of edge AI for scalability and privacy, and introduces Latent AI's technology for compressing AI models (e.g., MobileNetV2) using quantization to 8-bit precision, achieving significant size reduction and performance improvements (e.g., 8x faster inference compared to Tensorflow.js). The post presents experimental results on memory and inference times for different model sizes and bit precisions, demonstrating the feasibility of running optimized AI models at the edge with low latency.

Jun 2020Using data science and machine learning for improved customer supportThis post details the application of data science and machine learning techniques to improve customer support. It covers three main areas: 1. Language Classification: Using a combination of classified language, Accept-Language header, and country information to improve accuracy, especially for short messages. 2. Fuzzy String Matching: Comparing algorithms like Levenshtein and Cosine similarity for extracting error messages from support tickets, opting for Cosine similarity due to its O(k+l) time complexity and exploring optimizations like tokenization and FastText. 3. Threat Identification: Using metrics like the ratio of 499 error codes to 5xx HTTP status codes for DDoS detection, and employing multi-dimensional data analysis (clustering) on HTTP headers to fingerprint credential stuffing attacks by correlating requests based on user agents and zones.

Feb 2019Stop the Bots: Practical Lessons in Machine LearningThis post details the application of machine learning for bot detection and prediction, outlining the stages of Business Intelligence, Standalone Machine Learning, and Machine Learning Productization. It highlights the use of CatBoost for training models on a large dataset of requests to predict bot attacks and credential stuffing, achieving a 99.68% true positive rate for WordPress login pages. Further development includes predicting which companies will be attacked next based on firmographic data.

What this trail covers