Giving users choice with Cloudflare’s new Content Signals Policy

What this post added

This post introduces the Content Signals Policy, a new addition to robots.txt that allows website operators to express preferences for how their content can be used after it has been accessed. It defines three content signals: 'search', 'ai-input', and 'ai-train', with machine-readable 'yes' or 'no' syntax. This policy aims to address the free-rider problem of data scraping and the "impossible choice" website operators face between open access and data misuse, particularly in the context of AI model training.