Long-Horizon Agents Need Experiments, Not Just Prompts — Erina Karati

This post introduces an autoresearch loop framework for improving the long-horizon behavior of AI agents. The framework addresses the breakdown of agent memory and decision-making over extended periods by wrapping agents in a controlled experimental loop. Key components include: running controlled scenarios (e.g., spreading facts, rumors), collecting detailed traces, evaluating performance on a balanced scorecard (reach, source retention, uncertainty preservation, replanning, privacy), and iteratively improving a small, frozen policy surface. The post argues that memory alone is insufficient for long-horizon agents and that rollback mechanisms and experimental validation are critical, extending these lessons beyond game agents to broader AI engineering applications.