Running a Software Factory Efficiently at Uber Scale

This post details Uber's "Software Factory" approach to efficiently running AI agents at scale. It breaks down AI usage into four layers and introduces a cost equation (users × sessions/user × turns/session × requests/turn × tokens/request × price/token) to measure and optimize spend. Key optimization levers discussed include benchmark-driven model selection for Pareto efficiency (cost/completed task, output quality, model reliability), default model selection for subagents to cost-effective models, and strategies to reduce tokens per request through defaults (400k token compaction, Medium reasoning effort) and prompt caching (optimizing TTL based on provider costs). The post also touches on managed agent outcomes like cost per merged PR and quality signals like revert rate.