Playbook Review Multi-Agent System

11 posts · Sep 2024 – Sep 2026

What this trail covers

Harvey's platform continues to evolve its AI agent capabilities, shifting the paradigm from prompt-based assistance to task-oriented execution. The system now empowers agents to autonomously plan, research, execute, and deliver finished work products, with lawyers focusing on goal setting and final review. This evolution spans transactional work, litigation, compliance, and in-house operations, aiming to increase leverage by automating repeatable, high-volume tasks. The platform now integrates m

All Harvey trails

Sep 2026Contract Review Agents That Turn Past Deals Into Better OutcomesThis post introduces Contract Review Agents, an evolution of the Playbook Review system, built as a multi-agent system. These agents are dynamic and learn from past deals, precedents, and negotiation guidelines. Key technical contributions include: 1. Capturing institutional knowledge through guided interviews to understand practical application of standards, including fallback ranges and context-dependent position changes. 2. Precedent benchmarking that compares new redlines against a bank of executed contracts, visualizing clause-by-clause comparisons. 3. A mechanism to keep review standards current by parsing completed deal data to identify divergences from agent standards and proactively recommending updates.
Sep 2026Post-Training RLM Agents for End-to-End M&A DiligenceThis post details research on applying recursive language model (RLM) harnesses and post-training techniques to improve AI performance on complex M&A diligence tasks. It introduces the LAB Diligence environment with large data rooms (up to 80M tokens) and a detailed rubric for evaluation. The RLM harness, where a root agent delegates tasks to sub-agents within bounded context windows, significantly increases rubric criteria pass rates compared to standard tool-loop harnesses. The research demonstrates that post-training a Qwen3.5-122B-A10B orchestrator via reinforcement learning within the RLM harness boosted pass rates from 29.9% to 63.0%. Analysis of the division of labor shows that the root agent's coordination is critical, even when it uses fewer tokens than sub-agents. The post also discusses coverage and cost trade-offs, and highlights the potential for improving coordination through post-training the root agent.
Sep 2026Rebuilding Playbook Review as a Multi-Agent System | HarveyThis post details the architectural shift from a pipeline of prompts to a multi-agent system for Playbook Review. It introduces an orchestrator-worker pattern with specialized subagents that work on individual rules in parallel, each operating on a copy of the document. Key technical contributions include the use of unique identifiers for document elements to enable precise citation and editing, a branching mechanism to manage concurrent agent edits, and the development of an evaluation suite with legal judgment rubrics to measure risk classification and redline quality. The post also contrasts the limitations of the previous pipeline architecture with the benefits of the new agentic approach in handling complex conditional logic, inter-rule interactions, and deal context.
Aug 2026Introducing Harvey IIThis post introduces Harvey II, which enhances agent capabilities by providing persistent context and memory. Agents now inherit the context of their 'Space' (matter/project), including documents, parties, tasks, permissions, and history. This eliminates the need for lawyers to re-establish context for each task. 'Memory' allows agents to learn and apply user preferences and work patterns, reducing repetitive instructions and corrections. The introduction of 'Harvey Tenet,' a legal-specific model, aims to improve reasoning and cost-effectiveness for continuous agent operation. The system also synchronizes permissions and ethical walls from existing systems and ties usage/cost to specific matters.
Aug 2026Turn Your Standards Into Stronger Reviews | HarveyIntroduces a new playbook builder that leverages existing standards, past documents, or guided conversations to create review-ready playbooks. The system analyzes sources, asks clarifying questions, and drafts rules in real-time, displaying the playbook's evolution. It captures preferred positions, fallback language, guidance, actions, conditions, and escalation paths. Quality controls are integrated, including surfacing coverage gaps and suggesting improvements. Rules are enriched with summaries, explanations, and citations. The system allows for updating specific playbook elements like fallbacks or liability caps without rebuilding rules manually.
Advertisement
Jun 2026How AI agents are changing the way lawyers do legal workThis post details the architectural shift in legal AI from prompt-centric tools to agent-driven task execution. It outlines the five stages of an agent's workflow (Plan, Research, Work, Deliver, Review) and categorizes agent deployment into ad hoc, pre-built, and custom agents. The contribution highlights how agents are changing the nature of legal work by automating repeatable tasks in transactional, litigation, compliance, and in-house operations, and discusses the implications for associate training and AI governance.
Apr 2026Two Types of Legal Work, One Agentic Platform | HarveyThis post introduces a distinction between two types of legal work handled by Harvey's agentic platform: repeatable work (e.g., contract checklists, regulatory monitoring, term extraction) and complex, long-horizon work (e.g., analyzing contract suites, drafting memos, end-to-end diligence). For repeatable work, agents follow structured, consistent steps at scale. For complex work, agents collaborate with lawyers by generating plans for review, pausing for high-stakes decisions, and providing a transparent record of work and judgment calls. This evolution is driven by foundation models' improved ability to sustain complex reasoning over longer tasks and produce higher-quality outputs with less variance. The platform's infrastructure, governance, and compounding knowledge support both modes of execution.
Mar 2026Introducing Agent Builder: Build Smarter Agents for Complex Legal WorkThis post introduces Agent Builder, an evolution of Workflow Builder, enabling users to create and share custom AI agents for complex legal tasks. It highlights the ability of these agents to infer context, handle multi-step tasks more efficiently, and incorporate human-in-the-loop checkpoints for control. The post also mentions future capabilities like scheduling agents for background execution and the development of long-horizon agents.
Nov 20253 Principles That Helped us Scale Agent DevelopmentThis post details the adoption of an agent framework for scaling AI feature development at Harvey. It introduces three core principles: "No Custom Orchestration" (using external SDKs like OpenAI Agent SDK instead of bespoke orchestration), "Capabilities are Tool Bundles" (packaging multiple tools or sub-agents into a single, portable entity that can inject instructions into the system prompt), and "Eval Gates on Capabilities" (requiring datasets and evaluators for Tool Bundles and system prompt upgrades to pass leave-one-out gates, preventing conflicts and context rot). The post highlights the shift from traditional code merging to "merging English" and the challenges of shared ownership of a single AI brain.
Jun 2025Design Customizable Workflows With Agent Builder | HarveyThis post announces the general availability of Agent Builder (formerly Workflow Builder), a self-serve tool for legal teams to design and deploy custom AI agents that encapsulate firm-specific knowledge, tone, and processes. It allows for building workflows from plain language descriptions or a no-code interface, customizing them with inputs, precedent examples, and branching logic, and then deploying, sharing, and refining them. The development was a collaboration with several legal firms.
Mar 2025Introducing Agents in Harvey | HarveyThis post introduces the concept of 'Agents' as an evolution beyond 'Model Systems' at Harvey. Agents are defined as model systems capable of planning, adapting, and interacting with humans. The post details the introduction of 'Assistant Workflows,' which are client-facing instantiations of agents designed to guide users through tasks. Key features include personalized workflow recommendations, guided task execution, transparent 'thinking states,' and expert-quality outputs optimized for specific use cases. Custom evaluations comparing workflow outcomes against human lawyers are also highlighted, showing performance at or above human lawyer-level for structured drafting and analysis tasks, and preferred outputs for unstructured tasks.
Sep 2024Harvey is building legal agents and workflows with OpenAI o1This post details Harvey's evaluation and integration strategy for OpenAI's new o1 models. It highlights the use of objective benchmarks (BigLaw Bench) and expert preference evaluations to assess model performance. The key technical contribution is the focus on "step-change" review to identify novel functionalities in agentic capabilities like reasoning, planning, and collaboration. The post discusses how Harvey aligns these generalist model capabilities with domain-specific legal reasoning by supplementing foundation models with legal and process data, and outlines the development of collaborative agents that can iteratively solve complex, underspecified problems like drafting an S-1.