Artificial Intelligence Integration & Impact
A deeper look at AI crawlers: breaking down traffic by purpose and industry

A deeper look at AI crawlers: breaking down traffic by purpose and industry

8/28/2025 · David Belson

What this post added

This post introduces new capabilities to the AI Insights page on Cloudflare Radar: industry-focused AI bot traffic data and a breakdown of AI bot traffic by purpose (Training, Search, User action, Undeclared). It details how these insights are presented, including crawl/refer ratios, traffic trends by purpose and industry, and how users can explore this data via Data Explorer with filters for vertical and industry.

Read the original post ↗