Using Agents to Scale Harvey’s Knowledge Sources

This post details the creation of 'The Data Factory,' an autonomous pipeline of AI agents designed to scale Harvey's knowledge sources. Key technical contributions include: an 'Intake engine' with a Sourcing Agent for automated legal infrastructure mapping and a Legal Review Agent for accelerated compliance review (doubling attorney throughput); a declarative configuration layer replacing hand-tuned systems, allowing jurisdictions to be defined by domain lists, filter hierarchies, permissions, and agent instructions, enabling a single reasoning agent to access parameterized tools across jurisdictions; and a four-step evaluation pipeline (Answer-first scenario generation, Production simulation, Trace validation, Multi-agent quality assessment) using specialized agents (URL Classification, Citation Quality, Answer Quality, Presentation, Decision) to rigorously test agent performance on new sources without human-labeled datasets.