Identify AI model overuse with User Insights

This post introduces enhancements to Cloudflare's User Insights for AI Gateway, focusing on providing deeper context for AI usage. Key contributions include: 1. Model Overkill View: Identifies conversations where the selected model may be more capable than the task requires, enabling teams to investigate potential cost savings and model appropriateness. 2. Task Analysis: Categorizes conversations by work type (coding, research, summarization, etc.) to understand how AI is being used. 3. Turn Analysis: Tracks the number of exchanges per task to assess conversation length and potential inefficiencies. 4. Integration with Auto Router: These insights power the Auto Router (in beta) to automatically route requests to appropriate models based on task, cost, and model fit. 5. Classification Engine: Details the asynchronous Cloudflare Worker that processes AI Gateway logs to generate these signals, using Durable Objects for metadata and R2 for log bodies. The classification is out-of-path and does not add latency to user responses.