Consistency diffusion language models: Up to 14x faster inference without sacrificing quality

This post introduces Consistency Diffusion Language Models (CDLM), a post-training recipe that accelerates DLM inference. CDLM combines consistency-based multi-token finalization with block-wise KV caching, enabling a block-wise causal student model that can reuse KV caches for finalized blocks. It employs three training objectives: distillation loss for newly unmasked positions, consistency loss for still-masked positions, and an auxiliary masked-denoising loss. CDLM achieves up to 14.5x latency speedups on math and coding tasks by reducing refinement steps and enabling exact KV caching, while maintaining competitive accuracy. The analysis also covers system-level performance characteristics, showing CDLM occupies an intermediate regime between AR and vanilla DLMs in terms of arithmetic intensity.