TechPrufer
Get started
Scale AI
Scale AI logoScale AI · Feature trail

Agent Failure Diagnosis

3 posts · Jul 2026 – Sep 2026

What this trail covers

Insights Generator is a system designed to help engineers diagnose and fix failures in AI agents by analyzing their full run history. It identifies failure patterns, quantifies their occurrence, and provides specific trace evidence, significantly improving engineer productivity in debugging and enhancing agent performance. This post details the system's rollout to Scale GenAI Platform customers as a diagnostic layer for production agents, enabling continuous improvement loops by surfacing an inc This post extends the Agent Failure Diagnosis capability by introducing a framework for determining when human intervention is economically justified in an agentic workflow. It defines high-value human work by comparing the expected loss from an agent's error against the cost of human review, incorporating factors like credit value, agent confidence, and the asymmetry of potential losses (e.g., forgone credits vs. escalated invalid credits). This allows for a risk-adjusted threshold for human review, optimizing the allocation of limited human capacity to decisions with the greatest expected value.

All Scale AI trails
  1. Why You Need to Red Team Your Enterprise AIThis post details the application of AI red teaming to enterprise AI systems, extending beyond model-level testing to encompass the entire system architecture including prompts, context retrieval, tool integration, agent orchestration, and rule sets. It introduces a quantitative approach to measuring system failures by differentiating between adversarial and ordinary user behavior, highlighting that ordinary users can trigger violations almost as frequently as skilled attackers. The post quantifies failure rates across different personas and system layers, using a case study of a global professional services firm to demonstrate the effectiveness of a cyclical test-patch-retest methodology against a defined harm taxonomy. It emphasizes the importance of a written harm taxonomy with specific risk levels and response modes for effective testing and risk management, and introduces the concept of 'packaging' attacks where seemingly harmless inputs can be combined to exploit system vulnerabilities. The post also highlights the challenge of irreversible actions in enterprise systems and the need to score attempted reach as well as success.
  2. In An Agentic World Where Automation Gets Cheap, Which Work Is Worth Routing to a Human?Introduces a quantitative framework for determining the economic threshold for human intervention in agentic workflows, specifically applied to accounts payable credit recovery. The framework defines high-value human work as `P(error) × Impact(error) > Cost(review)`, where `P(error)` is derived from agent confidence and `Impact(error)` accounts for credit face value and a severity factor reflecting the asymmetry of losses (e.g., forgone credits vs. escalated invalid credits). This allows for risk-adjusted review thresholds, optimizing human review capacity towards decisions with the greatest expected value.
  3. Insights Generator: Diagnosing Agent FailuresThis post details the rollout of Insights Generator to Scale GenAI Platform customers, presenting it as a diagnostic layer for production agents. It highlights how the system analyzes an agent's full run history to identify, quantify, and provide trace evidence for failure patterns, aiming to shift agent improvement from a manual, one-off project to a continuous loop. The post includes specific examples of failure compound observed in benchmarks like AppWorld, SpreadsheetBench, and Humanity's Last Exam, demonstrating how Insights Generator surfaces issues that manual trace inspection might miss.