Trapping misbehaving bots in an AI Labyrinth
Mar 19, 2025Reid Tatoris, Harsh Saxena, Luis Miglietti
Summary
Introduced AI Labyrinth, a new bot mitigation technique that uses AI-generated content to trap and confuse AI crawlers. This involves pre-generating HTML pages with AI (Workers AI, open-source model), sanitizing them for XSS, and storing them in R2. These pages are then integrated as hidden links on existing pages via HTML transformation. The system identifies suspected AI scrapers by observing which bots follow these hidden links, feeding this data into machine learning models for improved bot identification. This feature is available as a simple toggle in the Cloudflare dashboard.
More from the Website Security & Threat Management trail
- We tested our own WAF with frontier AI models. Here’s what we found
- When scanners miss the attack: how Cloudflare Client-Side Security protects storefronts
- Introducing automatic remediation policies with Cloudflare CASB
- Introducing context-aware vulnerability discovery and remediation with Cloudflare Managed Defense and OpenAI Daybreak Models
- Introducing Adaptive Intelligence: undermining the economics of every bot attack