Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models

This post presents a comprehensive study of the performance characteristics of ARMs and DLMs, combining theoretical analysis with empirical profiling. It shows that DLMs can achieve higher arithmetic intensity than ARMs by leveraging parallelism across token positions but fail to scale effectively with longer contexts. It explores block-wise decoding for DLMs to enable better scaling to long contexts and examines batched inference, finding that ARMs exhibit superior throughput. Opportunities for accelerating DLM inference are highlighted, emphasizing that reducing the number of sampling steps is key for open-source DLMs to achieve lower latency relative to ARMs.