Consistency diffusion language models: Up to 14x faster inference without sacrificing quality

What this post added

This post introduces Consistency Diffusion Language Models (CDLM), a novel training-based acceleration scheme for Diffusion Language Models (DLMs). CDLM enables faster inference by combining consistency modeling with block-wise KV caching. Key technical contributions include: 1) A post-training recipe that makes fewer-step inference reliable while enabling exact block-wise KV caching. 2) Training objectives: distillation loss for newly unmasked positions, consistency loss for still-masked positions, and an auxiliary DLM masked-denoising loss. 3) A block-wise causal student architecture that attends to the prompt, previously completed blocks, and the current decoding block. 4) Inference strategy using block-wise autoregressive decoding with KV cache reuse and confidence-thresholded parallel finalization within blocks. The post also provides system-level analysis of arithmetic intensity and roofline placement for CDLM compared to AR and vanilla DLMs, explaining its efficiency at small batch sizes.