Make Your Website Conversational for People and Agents with NLWeb and AutoRAG

What this post added

Introduces NLWeb, an open standard for natural-language queries on websites, and integrates it with Cloudflare's AutoRAG. AutoRAG is enhanced with website crawling capabilities to index dynamic content from websites into R2 and Vectorize. A Cloudflare Worker is deployed to implement the NLWeb protocol, providing a conversational UI and an MCP server endpoint for AI agents. The AutoRAG architecture is updated with JobManager and FileManager Durable Objects to handle website crawling and improve scalability for parallel file processing.