Securing internal systems against increasingly capable and imperfectly aligned AI

This post details the development and application of a novel threat-modelling framework for AI agents, treating them as potential 'insider threats' by adapting the MITRE ATT&CK framework. It outlines a system for AI control mitigations, including monitoring using supervisor AI systems for detection, and prevention/response mechanisms. Key metrics for performance measurement are coverage, recall, and time-to-response. The post discusses scaling security protocols in response to increasing AI capabilities, specifically addressing evasion of detection (oversight awareness, opaque reasoning) and potential for harm (shifting from asynchronous to synchronous response). It highlights the analysis of one million coding agent trajectories to refine safety protocols and build live monitoring systems, identifying behavioral patterns and distinguishing between adversarial intent and agent misinterpretation. The technical framework 'Three Layers of Agent Security' is also introduced.