Catch Agent Regressions Before You Ship: Evals for Managed Deep Agents

This post details the evaluation framework for Managed Deep Agents, focusing on catching regressions and improving agent performance. It introduces the concept of evals as composed of an environment, job, and check, and demonstrates scaffolding with `mda evals init`. The post highlights the use of coding agents to generate eval tasks and LangSmith for monitoring and analysis, including datasets, traces, and rewards. It also presents simpler pytest-based evals and discusses production deployment via nightly CI.