
8/13/2025
What this post added
This post introduces and categorizes three types of AI bot traffic: AI training crawlers, AI engine grounding bots, and AI referrals. It explains the value of each type for discoverability and growth, and provides guidance on when and how to selectively block AI crawlers without harming discoverability. It also mentions Vercel's tools (Vercel Firewall, Bot Protection, BotID) as solutions for managing this traffic.