TechPrufer
Get started
Anthropic
Anthropic logoAnthropic · Feature trail

Prompt Injection Robustness

1 post · Apr 2024 – Nov 2025

What this trail covers

Anthropic is developing and iterating on methods to enhance the robustness of AI models, particularly Claude, against prompt injection attacks, especially in browser-based agent scenarios. This involves improving model training, classifier accuracy, and expert human red teaming to mitigate risks associated with untrusted web content. The goal is to enable AI agents to perform high-value tasks safely and reliably, even when interacting with the internet. Claude Opus 4.5 demonstrates stronger prompt injection robustness. This post introduces sandboxing for Claude Code, which includes filesystem and network isolation, built on OS-level primitives like Linux bubblewrap and MacOS seatbelt. This sandboxing reduces permission prompts by 84% and allows Claude to run more autonomously. A new sandbox runtime is open-sourced for broader adoption. Claude Code on the web also runs in an isolated sandbox, with sensitive credentials never inside the sandbox with Claude Code.

All Anthropic trails
  1. Mitigating the risk of prompt injections in browser useThis post details improvements in prompt injection robustness for Claude Opus 4.5, specifically in the context of browser-based AI agents. Key contributions include: 1. Reinforcement learning to directly train Claude to resist prompt injections by rewarding correct identification and refusal of malicious instructions. 2. Enhancements to classifiers that scan untrusted content for adversarial commands, improving detection of hidden text, manipulated images, and deceptive UI elements. 3. Scaled expert human red teaming, including participation in industry benchmark challenges, to discover creative attack vectors. The post also announces the expansion of Claude for Chrome from research preview to beta, indicating a more robust and safer user experience.
  2. Making Claude Code more secure and autonomous with sandboxingIntroduced sandboxing for Claude Code, featuring filesystem isolation (limiting access to specific directories) and network isolation (allowing connections only to approved servers). This is built on OS-level primitives like Linux bubblewrap and MacOS seatbelt. The sandboxing reduces permission prompts by 84% and allows Claude to run more autonomously. A new open-source sandbox runtime is provided for broader adoption. Claude Code on the web also utilizes an isolated sandbox, ensuring sensitive credentials are not exposed.
  3. Many-shot jailbreakingThis post introduces and details the 'many-shot jailbreaking' technique, which exploits long context windows in LLMs by using a large number of faux dialogues within a single prompt to bypass safety guardrails. It demonstrates that this technique scales with context window length and is often more effective on larger models. The post also discusses the relationship between many-shot jailbreaking and in-context learning, showing similar statistical patterns. It outlines mitigation strategies, including prompt-based classification and modification, which have shown significant success in reducing attack rates.