
8/5/2012 · Matthew Prince
What this post added
Introduced Always Online v.2, which actively crawls customer pages to build a more robust cache, addressing the limitations of the previous version that relied on search engine crawler traffic. This active crawling mechanism ensures that cached content is more consistently available when origin servers go offline. The crawler's user agent is Mozilla/5.0 (compatible; CloudFlare-AlwaysOnline/1.0; +http://www.cloudflare.com/always-online). Crawl frequency varies by plan: every 9 days for free customers, every 3 days for Pro, and daily for Business/Enterprise. Future plans include user control over pinning and excluding pages.