Your site, your rules: new AI traffic options for all customers

What this post added

Introduced a new pragmatic taxonomy for classifying automated traffic into three primary AI use cases: Search, Agent, and Training. This taxonomy allows for more granular control over bot access, moving beyond a simple 'AI bot' block. New defaults will be applied on September 15, 2026, blocking Training and Agent bots on pages displaying ads by default, while allowing Search bots. Multi-purpose crawlers will be subject to the most restrictive applicable rules. Launched BotBase for Enterprise customers, providing a searchable database of known bots and their classifications, with future plans for direct control. Introduced new 'content use' settings (immediate, reference, full) to further refine bot management based on how content is stored and reused. Extended Content Signals with a new `use` signal in robots.txt.