
6/26/2011 · Matthew Prince
What this post added
This post details how Cloudflare addressed an issue where search engine crawlers were reducing crawl rates for sites behind Cloudflare due to issues with a single origin server. Cloudflare developed a new technology to automatically change a site's Cloudflare IP addresses to isolate it from other sites when a problem is detected. They also collaborated with search engine crawl teams to ensure Cloudflare's IP ranges were categorized correctly, allowing search engines to crawl at maximum velocity without overburdening origin servers. This resulted in improved SEO rankings and page speed scores for users.