Robust Learning of Diffusion Models with Extremely Noisy Conditions

2025/10/11 by Xin Chen, Gillian Dobbie, Chen, Xin +9
Computer Science · Physics and Astronomy · #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2510.10149

openalex publication_date 2025/10/11 · openalex created_date 2025/10/15 · openalex updated_date 2026/07/28

Abstract

Conditional diffusion models have the generative controllability by incorporating external conditions. However, their performance significantly degrades with noisy conditions, such as corrupted labels in the image generation or unreliable observations or states in the control policy generation. This paper introduces a robust learning framework to address extremely noisy conditions in conditional diffusion models. We empirically demonstrate that existing noise-robust methods fail when the noise level is high. To overcome this, we propose learning pseudo conditions as surrogates for clean conditions and refining pseudo ones progressively via the technique of temporal ensembling. Additionally, we develop a Reverse-time Diffusion Condition (RDC) technique, which diffuses pseudo conditions to reinforce the memorization effect and further facilitate the refinement of the pseudo conditions. Experimentally, our approach achieves state-of-the-art performance across a range of noise levels on both class-conditional image generation and visuomotor policy generation tasks.The code can be accessible via the project page https://robustdiffusionpolicy.github.io

Citations

• Optimized Gradient Clipping for Noisy Label Learning
• On Improved Conditioning Mechanisms and Pre-training Strategies for Diffusion Models
• Don't drop your samples! Coherence-aware training benefits Conditional diffusion
• MOWA: Multiple-in-One Image Warping Model
• Hierarchical Diffusion Policy for Kinematics-Aware Multi-Task Robotic Manipulation
• Label-Noise Robust Diffusion Models
• DiffiT: Diffusion Vision Transformers for Image Generation
• StableVITON: Learning Semantic Correspondence with Latent Diffusion Model for Virtual Try-On
• DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models
• StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Models
• Manifold DivideMix: A Semi-Supervised Contrastive Learning Framework for Severe Label Noise
• Parallel Sampling of Diffusion Models
• AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation
• Conditional Image-to-Video Generation with Latent Flow Diffusion Models
• Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
• Diffusion policy: Visuomotor policy learning via action diffusion
• Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation
• Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labels
• Conditional Diffusion Based on Discrete Graph Structures for Molecular Graph Generation
• Semantic-Conditional Diffusion Networks for Image Captioning
• Inversion-Based Style Transfer with Diffusion Models
• DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
• A Survey on Generative Diffusion Model
• A Survey on Generative Diffusion Models
• Classifier-Free Diffusion Guidance
• Elucidating the Design Space of Diffusion-Based Generative Models
• Diffusion-LM Improves Controllable Text Generation
• Denoising Likelihood Score Matching for Conditional Score-based Data Generation
• Selective-Supervised Contrastive Learning with Noisy Labels
• Class-Aware Contrastive Semi-Supervised Learning
• High-Resolution Image Synthesis with Latent Diffusion Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• Learning with Noisy Labels by Efficient Transition Matrix Estimation to Combat Label Miscorrection
• Implicit Behavioral Cloning
• Learning with Noisy Labels via Sparse Regularization
• Understanding and Improving Early Stopping for Learning with Noisy Labels
• Towards Understanding Deep Learning from Noisy Labels with Small-Loss Criterion
• Diffusion Models Beat GANs on Image Synthesis
• Learning Noise Transition Matrix from Only Noisy Labels via Total Variation Regularization
• Score-Based Generative Modeling through Stochastic Differential Equations
• Denoising Diffusion Implicit Models
• Early-Learning Regularization Prevents Memorization of Noisy Labels
• Normalized Loss Functions for Deep Learning with Noisy Labels
• Denoising Diffusion Probabilistic Models
• Dual T: Reducing Estimation Error for Transition Matrix in Label-noise Learning
• Reliable Fidelity and Diversity Metrics for Generative Models
• Meta Label Correction for Noisy Label Learning
• Are Anchor Points Really Indispensable in Label-Noise Learning?
• Learning From Noisy Labels By Regularized Estimation Of Annotator Confusion
• Label-Noise Robust Generative Adversarial Networks
• Robustness of Conditional GANs to Noisy Labels
• QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
• Dimensionality-Driven Learning with Noisy Labels
• Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels
• Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels
• Learning to Reweight Examples for Robust Deep Learning
• cGANs with Projection Discriminator
• Decoupling "when to update" from "how to update"
• Robust Loss Functions under Label Noise for Deep Neural Networks
• Uncertainty-Aware Reinforcement Learning for Collision Avoidance
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Temporal Ensembling for Semi-Supervised Learning
• Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach
• Improved Techniques for Training GANs
• Learning with Symmetric Label Noise: The Importance of Being Unhinged
• U-Net: Convolutional Networks for Biomedical Image Segmentation

Related

• Learning Robust Diffusion Models from Imprecise Supervision
• ShiftDDPMs: Exploring Conditional Diffusion Models by Shifting Diffusion Trajectories
• Towards Faster Training of Diffusion Models: An Inspiration of A Consistency Phenomenon
• Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
• Bi-Noising Diffusion: Towards Conditional Diffusion Models with Generative Restoration Priors