Malicious Code Detection

6/2/2026

This post details the expansion of Datadog's BewAIre system to scan entire software packages and registries, moving beyond pull request analysis. The core technical contribution is the implementation of a two-stage evaluation pipeline: a 'filter' phase using a cost-effective LLM to quickly screen changes, and an 'investigation' phase employing an agentic system with tool access (GitHub APIs, osv.dev, SCA) for deeper analysis of suspicious cases. This resulted in a significant accuracy improvement (97.4% to 99.86%) by reducing false positives and introduced techniques like diff-chunking, LLM-as-judge, and agentic investigation with tools. It also addresses challenges like context window limits and the need for static checks against typosquatting domains.

3/9/2026

This post details how Datadog's BewAIre system, previously focused on malicious code in PRs, was enhanced to detect AI agent-driven attacks targeting open-source repositories and CI/CD workflows. It describes the identification of the hackerbot-claw AI agent and its attempts to exploit GitHub Actions and LLM-powered workflows. The post highlights the system's ability to validate existing defensive controls and the subsequent hardening of systems to address these new threats, including the integration of security signals into Cloud SIEM and incident response workflows.

10/21/2025

This post details the initial build of Datadog's LLM-powered system, nicknamed BewAIre, for detecting malicious pull requests in real-time. It covers the ingestion and preprocessing of PRs, the inference process using LLMs to reason about code intent, and the generation of security signals. The post highlights the importance of prompt engineering and data tuning for accuracy, the challenges and solutions for working with LLM context windows, and the continuous testing against real-world exploits. It also mentions the system's availability in Preview for Static Code Analysis (SAST) customers.