Rebuilding Playbook Review as a Multi-Agent System | Harvey

This post details the architectural shift from a pipeline of prompts to a multi-agent system for Playbook Review. It introduces an orchestrator-worker pattern with specialized subagents that work on individual rules in parallel, each operating on a copy of the document. Key technical contributions include the use of unique identifiers for document elements to enable precise citation and editing, a branching mechanism to manage concurrent agent edits, and the development of an evaluation suite with legal judgment rubrics to measure risk classification and redline quality. The post also contrasts the limitations of the previous pipeline architecture with the benefits of the new agentic approach in handling complex conditional logic, inter-rule interactions, and deal context.