Plug-and-Play Image Restoration with Deep Denoiser Prior
2020/08/31 by Kai Zhang, Yawei Li, Zhang, Kai +9 · 101 citations
Computer Science · Engineering · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Image and Video Processing (eess.IV) #cs.CV #eess.IV #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2008.13751
An extended version of IRCNN (CVPR17). Project page: https://github.com/cszn/DPIR
arxiv created 2021/07/12 · arxiv updated 2021/07/14
Abstract
Recent works on plug-and-play image restoration have shown that a denoiser can implicitly serve as the image prior for model-based methods to solve many inverse problems. Such a property induces considerable advantages for plug-and-play image restoration (e.g., integrating the flexibility of model-based method and effectiveness of learning-based methods) when the denoiser is discriminatively learned via deep convolutional neural network (CNN) with large modeling capacity. However, while deeper and larger CNN models are rapidly gaining popularity, existing plug-and-play image restoration hinders its performance due to the lack of suitable denoiser prior. In order to push the limits of plug-and-play image restoration, we set up a benchmark deep denoiser prior by training a highly flexible and effective CNN denoiser. We then plug the deep denoiser prior as a modular part into a half quadratic splitting based iterative algorithm to solve various image restoration problems. We, meanwhile, provide a thorough analysis of parameter setting, intermediate results and empirical convergence to better understand the working mechanism. Experimental results on three representative image restoration tasks, including deblurring, super-resolution and demosaicing, demonstrate that the proposed plug-and-play image restoration with deep denoiser prior not only significantly outperforms other state-of-the-art model-based methods but also achieves competitive or even superior performance against state-of-the-art learning-based methods. The source code is available at https://github.com/cszn/DPIR.
Cited by
- Stabilizing Deep Reconstruction Operators with Contractive Anchoring
- Trainable Nonexpansive Denoisers for Contractive Image Reconstruction
- RPG-VST: Robust Poisson-Gaussian Variance Stabilization for Blind RAW Denoising
- Self-Supervised Learning from Noisy and Incomplete Data
- Small, Bias-Free, Blind and Convolutional Denoiser: A compact ConvNeXt U-Net for blind Gaussian color-image denoising
- Score-Based Turbo Message Passing for Plug-and-Play Compressive Imaging
- DiffusionBrowser: Interactive Diffusion Previews via Multi-Branch Decoders
- CHEM: Estimating and Understanding Hallucinations in Deep Learning for Image Processing
- CARD: Correlation Aware Restoration with Diffusion
- From sparse recovery to plug-and-play priors, understanding trade-offs for stable recovery with generalized projected gradient descent
- Multi Task Denoiser Training for Solving Linear Inverse Problems
- 2-Shots in the Dark: Low-Light Denoising with Minimal Data Acquisition
- CircleFlow: Flow-Guided Camera Blur Estimation using a Circle Grid Target
- History-Augmented Contrastive Learning With Soft Mixture of Experts for Blind Super-Resolution of Planetary Remote Sensing Images
- Deep Parameter Interpolation for Scalar Conditioning
- Blind Adaptive Local Denoising for CEST Imaging
- Diffusion Model-Enhanced Environment Reconstruction in ISAC
- MRI Super-Resolution with Deep Learning: A Comprehensive Survey
- Bregman geometry-aware split Gibbs sampling for Bayesian Poisson inverse problems
- Equivariant Sampling for Improving Diffusion Model-based Image Restoration
- Integrating Reweighted Least Squares with Plug-and-Play Diffusion Priors for Noisy Image Restoration
- Image Restoration via Primal Dual Hybrid Gradient and Flow Generative Model
- A Distributed Plug-and-Play MCMC Algorithm for High-Dimensional Inverse Problems
- Clustering by Denoising: Latent plug-and-play diffusion for single-cell data
- Plugging Weight-tying Nonnegative Neural Network into Proximal Splitting Method: Architecture for Guaranteeing Convergence to Optimal Point
- Diffusion models for polarimetric reconstruction of circumstellar environments
- Tensor Completion via Monotone Inclusion: Generalized Low-Rank Priors Meet Deep Denoisers
- Learning Regularization Functionals for Inverse Problems: A Comparative Study
- NPN: Non-Linear Projections of the Null-Space for Imaging Inverse Problems
- Transforming Noise Distributions with Histogram Matching: Towards a Single Denoiser for All
- TDiff: Thermal Plug-And-Play Prior with Patch-Based Diffusion
- On Implicit Concave Structures in Half-Quadratic Methods for Signal Reconstruction
- Rasterized Steered Mixture of Experts for Efficient 2D Image Regression
- Net2Net: When Un-trained Meets Pre-trained Networks for Robust Real-World Denoising
- LVTINO: LAtent Video consisTency INverse sOlver for High Definition Video Restoration
- FLOWER: A Flow-Matching Solver for Inverse Problems
- Model-based deep learning enables time-resolved computational microscopy
- FlashEdit: Decoupling Speed, Structure, and Semantics for Precise Image Editing
- PRNU-Bench: A Novel Benchmark and Model for PRNU-Based Camera Identification
- LenslessMic: Audio Encryption and Authentication via Lensless Computational Imaging
- Analysis Plug-and-Play Methods for Imaging Inverse Problems
- Breaking the Weak Recovery Limit in Random Phase Retrieval with Learned Regularizers
- DICE: Diffusion Consensus Equilibrium for Sparse-view CT Reconstruction
- UTOPY: Unrolling Algorithm Learning via Fidelity Homotopy for Inverse Problems
- Flow Matching-Based Active Learning for Radio Map Construction with Low-Altitude UAVs
- Deep Learning Architectures for Medical Image Denoising: A Comparative Study of CNN-DAE, CADTra, and DCMIEDNet
- From the Gradient-Step Denoiser to the Proximal Denoiser and their associated convergent Plug-and-Play algorithms
- Multispectral CT Denoising via Simulation-Trained Deep Learning: Experimental Results at the ESRF BM18
- Stabilizing RED using the Koopman Operator
- Solving Imaging Inverse Problems Using Plug-and-Play Denoisers: Regularization and Optimization Perspectives
- HyDeFuse: Provably Convergent Denoiser-Driven Hyperspectral Fusion
- IDF: Iterative Dynamic Filtering Networks for Generalizable Image Denoising
- Weakly-Convex Regularization for Magnetic Resonance Image Denoising
- A Convergent Generalized Krylov Subspace Method for Compressed Sensing MRI Reconstruction with Gradient-Driven Denoisers
- Reverse Convolution and Its Applications to Image Restoration
- Similarity Matters: A Novel Depth-guided Network for Image Restoration and A New Dataset
- Multivariate Fields of Experts
- Lightweight Quad Bayer HybridEVS Demosaicing via State Space Augmented Cross-Attention
- Higher Order Regularization using Harmonic Eigenfunctions for Model-Based Reconstruction in Magnetic Particle Imaging
- How Diffusion Prior Landscapes Shape the Posterior in Blind Deconvolution
- Viscosity Stabilized Plug-and-Play Reconstruction
- MAP Estimation with Denoisers: Convergence Rates and Guarantees
- Learned Single-Pixel Fluorescence Microscopy
- I2I-PR: Deep Iterative Refinement for Phase Retrieval using Image-to-Image Diffusion Models
- Encryption and Authentication with a Lensless Camera Based on a Programmable Mask
- Enhancing Diffusion Model Stability for Image Restoration via Gradient Management
- COS2A: Conversion from Sentinel-2 to AVIRIS Hyperspectral Data Using Interpretable Algorithm With Spectral-Spatial Duality
- Physics Matters in PnP: Recovery Guarantees with the MMSE and NN Denoisers
- AI-based Environment-Aware XL-MIMO Channel Estimation with Location-Specific Prior Knowledge Enabled by CKM
- Near-Field Communications with Different Array Geometries: Rayleigh Distance, Channel Estimation, and Transmission Design
- Learning few-step posterior samplers by unfolding and distillation of diffusion models
- Physics informed guided diffusion for accelerated multi-parametric MRI reconstruction
- Single Image Inpainting and Super-Resolution with Simultaneous Uncertainty Guarantees by Universal Reproducing Kernels
- EAMamba: Efficient All-Around Vision State Space Model for Image Restoration
- Enhancing Image Restoration Transformer via Adaptive Translation Equivariance
- Visual-Instructed Degradation Diffusion for All-in-One Image Restoration
- MoiréXNet: Adaptive Multi-Scale Demoiréing with Linear Attention Test-Time Training and Truncated Flow Matching Prior
- Unsupervised Imaging Inverse Problems with Diffusion Distribution Matching
- Restarted contractive operators to learn at equilibrium
- Zero-Shot Solving of Imaging Inverse Problems via Noise-Refined Likelihood Guided Diffusion Models
- Structural Similarity-Inspired Unfolding for Lightweight Image Super-Resolution
- Convergent Complex Quasi-Newton Proximal Methods for Gradient-Driven Denoisers in Compressed Sensing MRI Reconstruction
- Multilook Coherent Imaging: Theoretical Guarantees and Algorithms
- Proximal Algorithm Unrolling: Flexible and Efficient Reconstruction Networks for Single-Pixel Imaging
- Plug-and-Play Posterior Sampling for Blind Inverse Problems
- Fast Trajectory-Independent Model-Based Reconstruction Algorithm for Multi-Dimensional Magnetic Particle Imaging
- Integrating Intermediate Layer Optimization and Projected Gradient Descent for Solving Inverse Problems with Diffusion Models
- PreP-OCR: A Complete Pipeline for Document Image Restoration and Enhanced OCR Accuracy
- DeepInverse: A Python package for solving imaging inverse problems with deep learning
- Linear Convergence of Plug-and-Play Algorithms with Kernel Denoisers
- Stochastic Orthogonal Regularization for deep projective priors
- Learning Cocoercive Conservative Denoisers via Helmholtz Decomposition for Poisson Inverse Problems
- Physen-Noise2Noise: Physics-Guided Self-Supervised Defocus Deblurring with Bias Correction under Low-Light Conditions
- A geometry-based deep equilibrium model for image restoration under multiplicative Gamma noise
- Multiscale Tensor Summation Factorization as a New Neural Network Layer (MTS Layer) for Multidimensional Data Processing
- A neural operator view on U-Nets for inverse imaging problems
- SteerMusic: Enhanced Musical Consistency for Zero-shot Text-guided and Personalized Music Editing
- PG-DPIR: An efficient plug-and-play method for high-count Poisson-Gaussian inverse problems
- VibrantLeaves: A principled parametric image generator for training deep restoration models
- Stochastic momentum ADMM for nonconvex and nonsmooth optimization with application to PnP algorithm
- Examining Joint Demosaicing and Denoising for Single-, Quad-, and Nona-Bayer Patterns
Related