2024/11/17 by Huang, William, Yifeng Jiang, Jiang, Yifeng +4 · 1 citation
Computer Science · Environmental Science · #Computer Vision and Pattern Recognition (cs.CV) #Distributed Sensor Networks and Detection Algorithms #FOS: Computer and information sciences #Groundwater flow and contamination studies #Machine Learning (cs.LG) #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.2411.10932
openalex publication_date 2024/11/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Diffusion models have demonstrated significant promise in various generative tasks; however, they often struggle to satisfy challenging constraints. Our approach addresses this limitation by rethinking training-free loss-guided diffusion from an optimization perspective. We formulate a series of constrained optimizations throughout the inference process of a diffusion model. In each optimization, we allow the sample to take multiple steps along the gradient of the proxy constraint function until we can no longer trust the proxy, according to the variance at each diffusion level. Additionally, we estimate the state manifold of diffusion model to allow for early termination when the sample starts to wander away from the state manifold at each diffusion step. Trust sampling effectively balances between following the unconditional diffusion model and adhering to the loss guidance, enabling more flexible and accurate constrained generation. We demonstrate the efficacy of our method through extensive experiments on complex tasks, and in drastically different domains of images and 3D motion generation, showing significant improvements over existing methods in terms of generation quality. Our implementation is available at https://github.com/will-s-h/trust-sampling.