Crawler Hints supports Microsoft’s IndexNow in helping users find new content

What this post added

Introduced Crawler Hints, a feature that signals content changes to search engines via the IndexNow protocol. This aims to optimize search engine crawling by reducing wasted crawl traffic and its associated environmental impact. The feature leverages cache data and origin status codes to inform indexers about content updates. It is enabled via a simple toggle in the Cloudflare dashboard and is free for all customers. Future plans include defaulting Crawler Hints on for all customers and incorporating additional signals like etags, last-modified headers, and content hash differences.