Multi-token Residual Prediction | Modal Blog

What this post added

Introduced Multi-Token Residual Prediction (MRP) for diffusion language models (DLMs). MRP trains a small module to predict the residual between adjacent denoising steps, rather than the full next-step distribution. This enables DLMs to achieve speedups in static denoising regimes (e.g., speculative decoding with up to 1.56x throughput) and recover quality lost in dynamic denoising regimes (up to +16 accuracy points). The post details the challenges of direct MTP in DLMs and the architectural changes for MRP, including a frozen DLM backbone and a trained MRP module. It also outlines applications in speculative and direct decoding, with performance metrics on various benchmarks.