DeepSeek-V4 Flash 0731 vs GPT-5.6 Luna on DeepSWE: Cost and Coding

What this post added

This post introduces a cost-performance analysis for coding tasks, comparing DeepSeek-V4 Flash 0731 and GPT-5.6 Luna. It details their performance metrics (pass@1, pass@k), cost per rollout, failure modes, and domain/language-specific strengths. A key contribution is the "cascade" strategy, where DeepSeek-V4 Flash is used as a first stage, escalating to GPT-5.6 Luna only on failure. This strategy achieves higher accuracy (78.9%) than GPT-5.6 Luna alone (67.2%) at a lower cost ($0.385 vs $0.61 per task). The analysis highlights DeepSeek's cost-effectiveness and clean failure profile, and GPT-5.6 Luna's superior general coding capability, particularly in reasoning-heavy tasks.