Cyber evaluations of Claude 4

This post details the results of rigorous cyber evaluations conducted on Claude 4 Opus and Sonnet, in partnership with Pattern Labs. The evaluations involved standalone CTF challenges and complex network environment simulations to assess the models' cyber offense capabilities. Key findings include marked improvements in flexible thinking, adaptability to challenges, vulnerability identification, and the execution of complex multi-step attack chains. However, limitations were identified in maintaining coherent, long-horizon plans when presented with unexpected obstacles. These findings directly inform Anthropic's ongoing safety work and the refinement of AI-resistant technical evaluation designs.