WebCode: Search Evals for Coding Agents

This post introduces WebCode, a new set of evaluations for coding agents focused on web search quality. It details a novel methodology for evaluating content extraction against a golden reference derived from rendered screenshots and DOM, and for assessing retrieval quality using a discriminative 'groundedness' metric rather than solely generative correctness. The evaluation framework addresses the limitations of existing benchmarks by focusing on tasks where parametric memory fails and emphasizes the importance of precise, up-to-date information for agentic coding tasks.