Arbitrage: Efficient Reasoning via Advantage-Aware Speculation

What this post added

Introduces ARBITRAGE, a step-level speculative generation framework that uses an advantage-aware router to dynamically decide whether to accept a step from a draft model or regenerate with a target model. This differs from prior methods by not relying on fixed acceptance thresholds and by training a lightweight router to predict the relative advantage of the target model, thereby reducing unnecessary rejections and wasted target compute.