BigLaw Bench Deep Dive: Retrieval | Harvey

This post details the expansion of the BigLaw Bench to include retrieval-focused benchmarks. It presents an evaluation of Harvey's proprietary retrieval system against common embedding-based and reranking methods, demonstrating a 30% improvement in identifying relevant content. The post highlights the unique challenges of legal retrieval (contracts, discovery emails, research databases) and attributes Harvey's superior performance to the incorporation of metadata, task-specific features (like recency), and LLM-based reasoning for relevancy judgments. It also discusses the importance of domain- and task-specific optimization for retrieval efficacy.