Improving Diffusion Models for Inverse Problems Using Optimal Posterior Covariance

2024/02/03 by Xinyu Peng, Peng, Xinyu, Ziyang Zheng +11 · 7 citations
Mathematics · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Numerical methods in inverse problems

paper · pdf · doi:10.48550/arxiv.2402.02149

openalex publication_date 2024/02/03 · openalex created_date 2024/02/07 · openalex updated_date 2026/07/28

Abstract

Recent diffusion models provide a promising zero-shot solution to noisy linear inverse problems without retraining for specific inverse problems. In this paper, we reveal that recent methods can be uniformly interpreted as employing a Gaussian approximation with hand-crafted isotropic covariance for the intractable denoising posterior to approximate the conditional posterior mean. Inspired by this finding, we propose to improve recent methods by using more principled covariance determined by maximum likelihood estimation. To achieve posterior covariance optimization without retraining, we provide general plug-and-play solutions based on two approaches specifically designed for leveraging pre-trained models with and without reverse covariance. We further propose a scalable method for learning posterior covariance prediction based on representation with orthonormal basis. Experimental results demonstrate that the proposed methods significantly enhance reconstruction performance without requiring hyperparameter tuning.

Cited by

• Softly Constrained Denoisers for Diffusion Models Applied to Partial Differential Equations
• Inverse problems with diffusion models: MAP estimation via mode-seeking loss
• Jacobian-Aware Posterior Sampling for Inverse Problems
• Latent Refinement via Flow Matching for Training-free Linear Inverse Problem Solving
• PriorGuide: Test-Time Prior Adaptation for Simulation-Based Inference
• Diffusion models for inverse problems
• Diffusion models for multivariate subsurface generation and efficient probabilistic inversion

Related

• Enhancing Diffusion Models for Inverse Problems with Covariance-Aware Posterior Sampling
• Gaussian is All You Need: A Unified Framework for Solving Inverse Problems via Diffusion Posterior Sampling
• Unleashing the Denoising Capability of Diffusion Prior for Solving Inverse Problems
• Diffusion Model Based Posterior Sampling for Noisy Linear Inverse Problems
• Efficient Diffusion Posterior Sampling for Noisy Inverse Problems