Insights Generator: Diagnosing Agent Failures

This post details the rollout of Insights Generator to Scale GenAI Platform customers, presenting it as a diagnostic layer for production agents. It highlights how the system analyzes an agent's full run history to identify, quantify, and provide trace evidence for failure patterns, aiming to shift agent improvement from a manual, one-off project to a continuous loop. The post includes specific examples of failure compound observed in benchmarks like AppWorld, SpreadsheetBench, and Humanity's Last Exam, demonstrating how Insights Generator surfaces issues that manual trace inspection might miss.