Limits of Confidence in Diffusion

This post analyzes the theoretical limitations of confidence in discrete diffusion models. It proves that a step matches the training distribution only when the positions it writes are conditionally independent given the tokens already fixed. The authors show that no product of per-position distributions can match a dependent group and that per-position distributions do not determine group dependency. Empirical verification on ScanAndAdd shows generated distributions are 29x the sampling-noise floor total variation, while per-sample metrics are 1.0.