Continually improving our agent harness

This post details ongoing improvements to the Cursor agent harness, focusing on evolving the context window from static to dynamic, implementing robust measurement strategies (benchmarks, online experiments, keep rate, user satisfaction), and tracking/repairing degradations with detailed error classification and anomaly detection. It also covers customizing the harness for different models (e.g., OpenAI vs. Anthropic) and facilitating mid-chat model switching by adapting to different tool sets and conversation histories. Key technical contributions include the shift to dynamic context discovery, the use of 'Keep Rate' and LLM-based user satisfaction for evaluation, detailed error classification for tool calls (InvalidArguments, UnexpectedEnvironment, ProviderError), anomaly detection for expected errors, and custom instructions for mid-chat model switching.