How to Build a Model Router in the Harness

This post details the implementation of a model router within LangChain's Open SWE coding agent harness. The router dynamically selects models based on task mix, identified through LangSmith traces, aiming to optimize cost-performance. The implementation involved mapping task types, selecting models along a Pareto frontier, building the router logic within the harness (including swapping a classifier to Jev for faster tier tasks), and tracking outcomes via evals or A/B tests. An A/B test demonstrated a 64% reduction in median cost per thread with no measurable quality degradation. Future work includes benchmarks, subagent integration, and mid-thread rerouting.