Why I Built Fugue: Controlled Experiments for Agent Systems

This post introduces the Fugue framework for building and evaluating agent systems. It defines an agent system as comprising six parts: model, harness, tools, policy, memory, and runtime. Fugue enables controlled experiments by holding the workload and evaluator constant while systematically varying components of the agent system. The author details four engineering habits that improve individual agent runs: placing authoritative context near code, naming outcomes/constraints/verification/stop conditions, requiring receipts (evidence) for success, and testing tool exposure. The post highlights that the 'harness' (the execution path and run mechanics) significantly impacts agent performance, demonstrating a 23-percentage-point gap between two different harness implementations. This work shifts the focus from solely optimizing prompts to understanding and rigorously testing the entire agent system.