GLM-5.3 vs. GLM-5.3 Flash on DeepSWE: Cost, Coding, and Routing

This post details a comparative analysis between GLM-5.3 and GLM-5.3 Flash on the DeepSWE benchmark. It quantifies the performance (pass@1, pass@k) and cost differences, showing that GLM-5.3 Flash is 17x cheaper per rollout and faster, with a minimal drop in best-of-k quality (2.6 points at pass@4). The analysis reveals that distillation primarily impacts consistency and the ability to recover from flaky attempts, rather than core capability. A cascade strategy is proposed, running GLM-5.3 Flash first and escalating to GLM-5.3 for rejected answers, achieving 80.9% task completion at $1.70 per task. A key regression identified is the increased baseline test breakage rate for GLM-5.3 Flash (6.9% vs 4.4%).