How Agentic Search Unlocks Legal Research Intelligence at Harvey

This post details the implementation of 'Agentic Search,' a system that moves beyond naive RAG to an iterative reasoning process for legal research. Key technical contributions include: 1. **Agentic Search Workflow:** An implementation inspired by ReAct, interleaving reasoning and tool calls for query understanding, dynamic tool selection, retrieval, synthesis, completeness checks, and citation-backed responses. 2. **Addressing Key Challenges:** Solutions for 'Source Selection in the Wild' (improving tool selection precision to 0.8-0.9) and 'Scaling Tool Calls to Query Complexity' (calibrating agent effort to scale to 3-10 retrieval operations for complex queries). 3. **Privacy-Preserving Evaluation:** A data flywheel involving expert query generation, rapid prototyping, systematic evaluation using LangSmith traces, and targeted improvements to tool design, system prompts, and tool docstrings. This process enabled significant improvements in accuracy and efficiency for complex, multi-source legal queries.