Introducing AutoPatchBench: A Benchmark for AI-Powered Security Fixes
Apr 29, 2025TJ Byun, Cornelius Aschermann, Kai Yuan Thng, Wu Zhou, Yupeng Yang, Lauren Deason, Joshua Saxe
Summary
Introduced AutoPatchBench, a benchmark for evaluating AI program repair systems specifically for fuzzing-identified C/C++ vulnerabilities. This benchmark includes 136 real-world vulnerabilities with verified fixes sourced from the ARVO dataset, and incorporates automated verification through fuzzing and white-box differential testing. The post details the challenges of existing datasets and the selection criteria and verification process used to create AutoPatchBench.
More from the Application Security and Permissions trail
- How We’re Building Scam Alert on WhatsApp With End-to-End Encryption and Verifiability Guarantees
- Privacy-Aware Infrastructure in the AI-Native Era: An Asset Classification Case Study
- Labyrinth 1.1: Making End-to-End Encrypted Backups Even More Reliable
- How Meta Is Strengthening End-to-End Encrypted Backups
- Post-Quantum Cryptography Migration at Meta: Framework, Lessons, and Takeaways