Introducing BigLaw Bench: Arena

This post introduces BigLaw Bench: Arena (BLB: Arena), an internal system for head-to-head evaluation of AI systems on legal tasks using scaled, pairwise comparison. It details the process of lawyers comparing two outputs, providing preferences and reasoned explanations. The system aggregates these votes into Elo scores for quantitative ranking and analyzes qualitative feedback into preference drivers (Alignment, Trust, Presentation, Intelligence). Recent results are presented comparing Harvey Assistant (built on GPT-5) against prior implementations and leading foundation models, demonstrating Harvey Assistant's superior performance and confirming intelligence as a key preference driver. The post also discusses the limitations of Elo scores and the importance of qualitative analysis.