DiffusionGemma: 4x faster text generation

Introduces DiffusionGemma, an experimental 26B MoE model that utilizes text diffusion for up to 4x faster text generation on GPUs by processing entire blocks of text simultaneously. Key technical contributions include a novel diffusion head for speed, bi-directional attention enabling parallel token processing (256 tokens at a time), and iterative self-correction. It highlights trade-offs in output quality compared to autoregressive models and its suitability for low-concurrency, local inference scenarios. It also details integration with various serving and fine-tuning tools.