Agent Security Testing and Prompt Injection Defense

5 posts · Jul 2026 – Oct 2026

What this trail covers

The platform now supports agentic experimentation loops, enabling AI agents to autonomously hypothesize, experiment, evaluate, and iterate on tasks. This includes structured experiment tracking via an API, leaderboards for performance comparison, idea branching for exploring different hypotheses, and history logging for maintaining context. The system is designed to leverage otherwise idle compute resources for continuous research and development. This post introduces AgentFlow, a novel framework for agent security testing and prompt injection defense, and discusses the transition from passive world modeling to active agents in physical AI.

All Lambda trails

Oct 2026From world models to active agents: the next step for physical AIThis post discusses the transition from passive world modeling to active agents in physical AI, highlighting the need for agents to sense, reason, plan, and act within their environments. It references research presented at the CVPR 2026 Workshop on World Models Meet Active Sensing and Closed-Loop Planning (WMAS), including work on RoboWM-Bench, GEM-4D, and SAW-Bench. The post emphasizes that generative models are becoming part of the action loop, forming a continuous cycle of sense-understand-predict-plan-act. It also touches upon agent security testing and prompt injection defense as critical aspects of developing intelligent physical AI systems.
Sep 2026Closing the loop: agentic evaluation for image editing foundation modelsIntroduces EdiVal-Agent, a framework for agentic evaluation of image editing foundation models. EdiVal-Agent decomposes evaluation into object-level tasks, maintains state across editing turns using an evolving object pool, and coordinates specialized AI models and visual tools to verify instruction following, content consistency, and visual quality. It demonstrates 81.3% agreement with human judgments for instruction following, outperforming existing automated metrics. The framework reveals that strong single-turn performance does not guarantee strong multi-turn performance and provides fine-grained insights into model failures.
Sep 2026ReviewGrounder: grounding AI-assisted peer review in evidenceThis post introduces ReviewGrounder, an AI-assisted peer review system that leverages specialized agents to ground reviews in evidence. It details a three-stage process: a Review Drafter, followed by three grounding agents (Literature Searcher, Insight Miner, Result Analyzer), and finally a Review Aggregator. The system aims to provide evidence-based reviews by verifying claims against the paper's content, prior work, and experimental results. It also highlights the underlying infrastructure needs for such agentic systems, emphasizing Lambda's role in providing GPU compute for their development and evaluation.
Aug 2026AgentFlow: when the agent's workflow learnsIntroduces AgentFlow, a novel framework that makes an agent's workflow trainable. It splits agents into four modules (planner, executor, verifier, generator) and uses Flow-GRPO (flow-based group refined policy optimization) to train the planner on-policy within the live multi-turn loop. Flow-GRPO addresses credit assignment by broadcasting end-of-task outcomes back to each turn, enabling tractable single-turn updates. AgentFlow, using a 7B model, demonstrates significant performance gains across benchmarks compared to larger proprietary systems, highlighting the importance of workflow learning over model scaling alone. The post also contrasts the effectiveness of in-the-flow training (Flow-GRPO) with conventional offline supervised fine-tuning.
Jul 2026Prompt injection doesn't care what your agent does for a livingThis post details the findings from the AgentBeats Security Arena, a prompt-injection competition. It identifies four key rhetorical shapes (46% of winning attacks) that transfer across different agent domains (e.g., expense approval, supply chain). It also highlights that short, conversational attacks (15% of wins) are effective and difficult to block by surface features. The analysis reveals that scenario volume is not a reliable measure of coverage, as different attack mechanisms can exist within a single scenario. Furthermore, wins and fails are often separated by a single word or minor formatting change, emphasizing the need to benchmark near-misses. The post advocates for organizing risk assessment around transferable mechanisms rather than just domain coverage and suggests separating trusted instructions from untrusted input, gating consequential actions behind explicit confirmation.
Advertisement
Jul 2026Keeping 100k battles of untrusted agent code in their laneThis post details the operational infrastructure for the AgentBeats competition, focusing on the challenges of running 100k battles of untrusted adversarial agent code on shared hardware. Key contributions include: implementing independent per-GPU vLLM replicas instead of tensor parallelism for easier dynamic scaling and fault isolation; using a single Caddy container as the sole network boundary for load balancing, health checks, and controlled egress (PyPI only); and a dynamic concurrency limiter that resizes at runtime to allow adding GPUs mid-round without dropping battles. It also describes the serverless AWS submission path and the read-only credentials used by the evaluation host to limit the blast radius of compromised systems. The post explicitly notes that the containment measures are not a secure sandbox but sufficient for the competition's threat model.