Implicit Regularization of the Deep Inverse Prior Trained with Inertia

2025/06/03 by Nathan Buskulic, Buskulic, Nathan, Jalal Fadili +3 · 1 citation
Engineering · Mathematics · #Advanced Numerical Analysis Techniques #FOS: Computer and information sciences #Machine Learning (cs.LG) #Numerical methods in inverse problems #Topology Optimization in Engineering

paper · pdf · doi:10.48550/arxiv.2506.02986

openalex publication_date 2025/06/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Solving inverse problems with neural networks benefits from very few theoretical guarantees when it comes to the recovery guarantees. We provide in this work convergence and recovery guarantees for self-supervised neural networks applied to inverse problems, such as Deep Image/Inverse Prior, and trained with inertia featuring both viscous and geometric Hessian-driven dampings. We study both the continuous-time case, i.e., the trajectory of a dynamical system, and the discrete case leading to an inertial algorithm with an adaptive step-size. We show in the continuous-time case that the network can be trained with an optimal accelerated exponential convergence rate compared to the rate obtained with gradient flow. We also show that training a network with our inertial algorithm enjoys similar recovery guarantees though with a less sharp linear convergence rate.

Citations

• Inertial Methods with Viscous and Hessian driven Damping for Non-Convex Optimization
• Recovery Guarantees of Unsupervised Neural Networks for Inverse Problems trained with Gradient Descent
• Deep Internal Learning: Deep Learning from a Single Input
• Convergence and Recovery Guarantees of Unsupervised Neural Networks for Inverse Problems
• Deep learning: a statistical viewpoint
• Deep Learning Techniques for Inverse Problems in Imaging
• BP-DIP: A Backprojection based Deep Image Prior
• The troublesome kernel -- On hallucinations, no free lunches and the accuracy-stability trade-off in inverse problems
• Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal\n and Image Processing
• Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing
• First-order optimization algorithms via inertial systems with Hessian\n driven damping
• Towards moderate overparameterization: global convergence guarantees for training shallow neural networks
• Fine-Grained Analysis of Optimization and Generalization for\n Overparameterized Two-Layer Neural Networks
• Overparameterized Nonlinear Learning: Gradient Descent Takes the Shortest Path?
• Image Restoration using Total Variation Regularized Deep Image Prior
• Gradient Descent Provably Optimizes Over-parameterized Neural Networks
• Neural Tangent Kernel: Convergence and Generalization in Neural Networks
• Deep Image Prior
• Adam: A Method for Stochastic Optimization

Cited by

• Physics Matters in PnP: Recovery Guarantees with the MMSE and NN Denoisers

Related

• Recovery Guarantees of Unsupervised Neural Networks for Inverse Problems trained with Gradient Descent
• Convergence and Recovery Guarantees of Unsupervised Neural Networks for Inverse Problems
• Convergence Guarantees of Overparametrized Wide Deep Inverse Prior
• Deep neural networks can stably solve high-dimensional, noisy, non-linear inverse problems
• Data-consistent neural networks for solving nonlinear inverse problems