Building Engine v2: How LangChain approaches agent improvement

This talk introduces Engine v2, an evolution of LangChain's agent engineering framework. Engine v2 introduces capabilities for detecting cross-trace metric trends and inefficient agent paths, moving beyond single-trace analysis. It also adds validated fixes, which reproduce bugs and test their resolutions, and red teaming for pre-release agent testing. The talk highlights the importance of per-task benchmarks for complex agents and discusses the trade-offs of using smaller models for simpler tasks. It also previews Engine v3 plans, including BYOIP support for self-hosted Engine and the ability to propose cheaper models based on evals.