Giving users choice with Cloudflare’s new Content Signals Policy
Sep 24, 2025Will Allen
Summary
Introduces the Content Signals Policy, an extension to robots.txt that allows website operators to specify preferences for how their content can be used by crawlers. Defines three content signals: 'search', 'ai-input', and 'ai-train'. Provides human-readable explanations and machine-readable syntax for expressing these preferences. Cloudflare will automatically update managed robots.txt files for customers to disallow AI training by default. The policy is released under CC0 license to encourage adoption.
More from the Website Security & Threat Management trail
- We tested our own WAF with frontier AI models. Here’s what we found
- When scanners miss the attack: how Cloudflare Client-Side Security protects storefronts
- Introducing automatic remediation policies with Cloudflare CASB
- Introducing context-aware vulnerability discovery and remediation with Cloudflare Managed Defense and OpenAI Daybreak Models
- Introducing Adaptive Intelligence: undermining the economics of every bot attack