1,000 Agent Tasks in a Sandbox: What Breaks When LLMs Run Code — Kevin Orellana, AWS

This talk details the practical failures encountered when an AI coding agent was used to stress-test a code sandbox and cloud browser environment. It categorizes these failures into interface-layer bugs (e.g., image encoding, null bytes, unreadable files) and sandbox-layer limitations (e.g., memory exhaustion, file permissions, seccomp, eBPF). The contribution lies in demonstrating how an agent can be used to fuzz-test sandboxes and identify weaknesses in agentic execution environments, emphasizing the need for structured logging (SQLite) and the distinction between platform-specific issues and the underlying sandbox primitives. It also touches on the limitations of LLM-generated tests and the differences between local and remote agent environments.