Stochastic gradient descent based variational inference for infinite-dimensional inverse problems

2025/06/10 by Sui, Jiaming, Jia, Junxiong, Li, Jinglai
#FOS: Mathematics #Numerical Analysis (math.NA)

paper · doi:10.48550/arxiv.2506.08380

Abstract

This paper introduces two variational inference approaches for infinite-dimensional inverse problems, developed through gradient descent with a constant learning rate. The proposed methods enable efficient approximate sampling from the target posterior distribution using a constant-rate stochastic gradient descent (cSGD) iteration. Specifically, we introduce a randomization strategy that incorporates stochastic gradient noise, allowing the cSGD iteration to be viewed as a discrete-time process. This transformation establishes key relationships between the covariance operators of the approximate and true posterior distributions, thereby validating cSGD as a variational inference method. We also investigate the regularization properties of the cSGD iteration and provide a theoretical analysis of the discretization error between the approximated posterior mean and the true background function. Building on this framework, we develop a preconditioned version of cSGD to further improve sampling efficiency. Finally, we apply the proposed methods to two practical inverse problems: one governed by a simple smooth equation and the other by the steady-state Darcy flow equation. Numerical results confirm our theoretical findings and compare the sampling performance of the two approaches for solving linear and non-linear inverse problems.

Citations

• Non-centered parametric variational Bayes' approach for hierarchical inverse problems of partial differential equations
• Variational Inverting Network for Statistical Inverse Problems of Partial Differential Equations
• Stein variational gradient descent on infinite-dimensional space and applications to statistical inverse problems
• Convergence Rates of Variational Posterior Distributions
• Advances in Variational Inference
• Stochastic Gradient Descent as Approximate Bayesian Inference
• A hybrid adaptive MCMC algorithm in function spaces
• Hierarchical Bayesian Level Set Inversion
• Variational Inference: A Review for Statisticians
• On an adaptive preconditioned Crank-Nicolson MCMC algorithm for infinite dimensional Bayesian inferences
• Algorithms for Kullback-Leibler Approximation of Probability Measures in\n Infinite Dimensions
• A computational framework for infinite-dimensional Bayesian inverse problems. Part I: The linearized case, with application to global seismic inversion
• The Bayesian Approach To Inverse Problems
• Gradient Flow from a Random Walk in Hilbert Space

Related

• Stein variational gradient descent on infinite-dimensional space and applications to statistical inverse problems
• Functional normalizing flow for statistical inverse problems of partial differential equations
• Consistency of Variational Inference for Nonlinear Inverse Problems of Partial Differential Equations
• Stochastic Inverse Problem: stability, regularization and Wasserstein gradient flow
• VI-DGP: A variational inference method with deep generative prior for solving high-dimensional inverse problems