Introducing DataBench

This post introduces DataBench, a new benchmark designed to evaluate agentic analytics capabilities on realistic data work. It details the motivation for creating DataBench, contrasting it with existing benchmarks like Spider 2.0 and DABstep. The post presents the methodology, including the use of a synthetic business (Shorelane Commerce) and Hex's native Evals functionality. It shares performance results for various models (Opus 5, Claude Fable 5, GPT 5.6 Sol, GPT-5.6 Luna, Sonnet 5) across different task types (Q&A, open-ended delegated decisions, trap failure modes). Key findings include the high performance floor, the steep Pareto frontier for GPT-5.6 Luna, and the models' ability to handle messy data when intent is clear. It also identifies judgment as a key area where models struggle, particularly with trap failure modes requiring intuition and deeper reasoning.