Scaling malicious code detection from pull requests to the software supply chain | Datadog

What this post added

This post details the expansion of Datadog's BewAIre system to scan entire software packages and registries, moving beyond pull request analysis. The core technical contribution is the implementation of a two-stage evaluation pipeline: a 'filter' phase using a cost-effective LLM to quickly screen changes, and an 'investigation' phase employing an agentic system with tool access (GitHub APIs, osv.dev, SCA) for deeper analysis of suspicious cases. This resulted in a significant accuracy improvement (97.4% to 99.86%) by reducing false positives and introduced techniques like diff-chunking, LLM-as-judge, and agentic investigation with tools. It also addresses challenges like context window limits and the need for static checks against typosquatting domains.