DeepSeek V4 Pro 0813 vs GPT-5.6 Sol on DeepSWE: Cost, Coding, and Routing

This post introduces a comparative analysis of DeepSeek V4 Pro 0813 and GPT-5.6 Sol on the DeepSWE benchmark, focusing on cost, coding performance, and routing strategies. It details performance metrics like pass@1, pass@k, cost per rollout, and failure modes for each model. A key contribution is the demonstration of a cascaded routing strategy where DeepSeek V4 Pro 0813 is used as a first-stage model, escalating to GPT-5.6 Sol only when necessary. This strategy achieves 83.0% task completion at $3.35 per task, outperforming GPT-5.6 Sol alone (72.7% at $8.37) and an oracle router (80.8%). The analysis highlights DeepSeek V4 Pro 0813's strengths in cost-effectiveness and reach with multiple attempts, while GPT-5.6 Sol excels in single-shot precision and speed. The post also details domain-specific and language-specific performance differences and provides a comprehensive data table of results.