TechPrufer
Get started
Vercel
Vercel logoVercel · Feature trail

Deepsec Security Harness

1 post · May 2026 – Jul 2026

What this trail covers

DeepsecBench evaluates AI models for cybersecurity vulnerability detection in code, providing metrics like recall, precision, cost, and total time. The cost of capable analysis is decreasing, with open-weight models closing the gap with frontier models. The benchmark runs on an open-source codebase.

All Vercel trails
  1. DeepsecBench: evaluating model performance in finding cybersecurity vulnerabilitiesThis post introduces DeepsecBench, a new benchmark for evaluating AI models' ability to find cybersecurity vulnerabilities in application code. It details the benchmark's methodology, including the use of a golden set of findings, recall-weighted F2 scoring, and median of three runs. It presents a leaderboard of model performance, highlighting cost and time trade-offs. The post also discusses the falling cost of capable analysis and provides guidance on building scanning programs across multiple models and running benchmarks via AI Gateway.
  2. Introducing deepsec: The security harness for finding vulnerabilities in your codebaseIntroduces deepsec, a security harness powered by coding agents that runs on user infrastructure to find vulnerabilities in large codebases. It uses Claude and Codex for tailored investigation, with a workflow including scan, investigate, revalidate, enrich, and export steps. It supports optional fanout to Vercel Sandboxes for parallel execution and can be customized with plugins. It functions with both specialized cyber models and off-the-shelf models.