Boost Inference Performance up to 15x on NVIDIA Blackwell Using DFlash Speculative Decoding | NVIDIA Technical Blog

Introduces DFlash, an open-source block diffusion model for speculative decoding, which significantly accelerates LLM inference on NVIDIA Blackwell GPUs by drafting entire token blocks in parallel. Benchmarks demonstrate up to 15x throughput improvement for gpt-oss-120b and nearly double interactivity for Llama 3.1 8B compared to EAGLE-3. DFlash is integrated into vLLM and SGLang, enabling adoption without application refactoring, and has released 20 model checkpoints on Hugging Face.