The Death of the Code Review: What the Data Actually Says — Laurie Voss, Arize AI
This post analyzes the impact of AI agents on software development workflows, specifically focusing on code generation and review. It presents data indicating that while AI agents dramatically increase code output (741%), the increase in shipped software is significantly lower (30%), identifying human review as the primary bottleneck. The talk critiques traditional code review effectiveness, citing studies showing reviewer performance degradation beyond 400 lines and the emergence of extremely large PRs from agents. It explores current industry practices, including OpenAI's zero-human-code product, mergeability benchmarks (SWE-bench), and the integration of review into tools like Cursor and GitHub Copilot. The post argues that mergeability itself is becoming a training signal for frontier models and that the human checkpoint is shifting towards production as the final arbiter, with prompt injection posing a challenge to automated reviewers. The core technical contribution is the analysis of how AI agents are forcing a re-evaluation and systemic rebuilding of the code review process, moving towards automated review harnesses and production-level validation.
More from the Agentic Research and Discovery Loops trail
- Build a Research Agent with Claude Managed Agents (Part 2 of 4) — Anthropic
- Scheduled Agents, Dreaming and Q&A: Claude Managed Agents (Part 3 of 4) — Anthropic
- Hill-Climbing Skills: Improve Agents Without Changing the Model — Shubhankar Srivastava, Browserbase
- Parameter Golf with AutoResearch — Vayum Arora, Zhengyao Jiang, Dixing Xu & Dhruv Srikanth, Weco AI
- From Zero to Leaderboard: Agent Evaluation — Wolfram Ravenwolf, Weights & Biases