The Design Space of Tri-Modal Masked Diffusion Models

Introduces tri-modal Masked Diffusion Models (MDMs) pretrained from scratch on text, image-text, and audio-text data. Systematically analyzes multimodal scaling laws, modality mixing ratios, noise schedules, and batch-size effects. Proposes a novel SDE-based reparameterization to decouple physical and logical batch sizes, eliminating the need for optimal batch size tuning. Pretrains a preliminary 3B model showcasing unified design capabilities across text generation, T2I, and T2S tasks.