Deep Spectral Prior

2025/05/26 by Yanqi Cheng, Cheng, Yanqi, Zhao, Xuxiang +8 · 1 citation
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Mathematics #Neural Networks and Applications #Numerical Analysis (math.NA)

paper · pdf · doi:10.48550/arxiv.2505.19873

openalex publication_date 2025/05/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/04

Abstract

We introduce the Deep Spectral Prior (DSP), a new framework for unsupervised image reconstruction that operates entirely in the complex frequency domain. Unlike the Deep Image Prior (DIP), which optimises pixel-level errors and is highly sensitive to overfitting, DSP performs joint learning of amplitude and phase to capture the full spectral structure of images. We derive a rigorous theoretical characterisation of DSP's optimisation dynamics, proving that it follows frequency-dependent descent trajectories that separate informative low-frequency modes from stochastic high-frequency noise. This spectral mode separation explains DSP's self-regularising behaviour and, for the first time, formally establishes the elimination of DIP's major limitation-its reliance on manual early stopping. Moreover, DSP induces an implicit projection onto a frequency-consistent manifold, ensuring convergence to stable, physically plausible reconstructions without explicit priors or supervision. Extensive experiments on denoising, inpainting, and deblurring demonstrate that DSP consistently surpasses DIP and other unsupervised baselines, achieving superior fidelity, robustness, and theoretical interpretability within a unified, unsupervised data-free framework.

Citations

• Efficient Attention-Sharing Information Distillation Transformer for Lightweight Single Image Super-Resolution
• You KAN Do It in a Single Shot: Plug-and-Play Methods with Single-Instance Priors
• PrefPaint: Aligning Image Inpainting Diffusion Model with Human Preference
• Sequential Diffusion-Guided Deep Image Prior For Medical Image Reconstruction
• Analysis of Deep Image Prior and Exploiting Self-Guidance for Image Reconstruction
• SinSR: Diffusion-Based Image Super-Resolution in a Single Step
• ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting
• WIRE: Wavelet Implicit Neural Representations
• Rethinking Deep Image Prior for Denoising
• NAS-DIP: Learning Deep Image Prior with Neural Architecture Search
• Time-Dependent Deep Image Prior for Dynamic MRI
• Region-wise Generative Adversarial ImageInpainting for Large Missing Areas
• A Bayesian Perspective on the Deep Image Prior
• DeepRED: Deep Image Prior Powered by RED
• Noise2Self: Blind Denoising by Self-Supervision
• Image Restoration using Total Variation Regularized Deep Image Prior
• On the Spectral Bias of Neural Networks
• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• Compressed Sensing with Deep Image Prior and Learned Regularization
• Free-Form Image Inpainting with Gated Convolution
• Deep Image Prior
• Globally and locally consistent image completion
• The Little Engine that Could: Regularization by Denoising (RED)
• Deep Residual Learning for Image Recognition
• U-Net: Convolutional Networks for Biomedical Image Segmentation
• Nonlinear total variation based noise removal algorithms
• An Algorithm for Total Variation Minimization and Applications

Cited by

• DNA-Prior: Unsupervised Denoise Anything via Dual-Domain Prior

Related

• On Measuring and Controlling the Spectral Bias of the Deep Image Prior
• The Spectral Bias of the Deep Image Prior
• Image Reconstruction via Deep Image Prior Subspaces
• Analysis of Deep Image Prior and Exploiting Self-Guidance for Image Reconstruction
• DeepRED: Deep Image Prior Powered by RED