2025/06/10 by Alan N. Amin, Nate Gruver, Amin, Alan N. +3 · 1 voice · 7 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · Mathematics · Medicine · #Advanced Neuroimaging Techniques and Applications #Cell Image Analysis Techniques #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Mathematical Biology Tumor Growth #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.2506.08316
openalex publication_date 2025/06/10 · arxiv published 2025/06/10 · arxiv updated 2025/09/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Discrete diffusion models, like continuous diffusion models, generate high-quality samples by gradually undoing noise applied to datapoints with a Markov process. Gradual generation in theory comes with many conceptual benefits; for example, inductive biases can be incorporated into the noising Markov process, and access to improved sampling algorithms. In practice, however, the consistently best performing discrete diffusion model is, surprisingly, masking diffusion, which does not denoise gradually. Here we explain the superior performance of masking diffusion by noting that it makes use of a fundamental difference between continuous and discrete Markov processes: discrete Markov processes evolve by discontinuous jumps at a fixed rate and, unlike other discrete diffusion models, masking diffusion builds in the known distribution of jump times and only learns where to jump to. We show that we can similarly bake in the known distribution of jump times into any discrete diffusion model. The resulting models - schedule-conditioned discrete diffusion (SCUD) - generalize classical discrete diffusion and masking diffusion. By applying SCUD to models with noising processes that incorporate inductive biases on images, text, and protein data, we build models that outperform masking.