2020/08/13 by Valentin De Bortoli, Alain Durmus, De Bortoli, Valentin +5
Engineering · Mathematics · #Computation (stat.CO) #FOS: Computer and information sciences #FOS: Mathematics #Numerical methods in inverse problems #Probability (math.PR) #Sparse and Compressive Sensing Techniques #Statistical Methods and Inference #Statistics Theory (math.ST)
paper · pdf · doi:10.48550/arxiv.2008.05793
openalex publication_date 2020/08/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
This paper presents a detailed theoretical analysis of the three stochastic approximation proximal gradient algorithms proposed in our companion paper [49] to set regularization parameters by marginal maximum likelihood estimation. We prove the convergence of a more general stochastic approximation scheme that includes the three algorithms of [49] as special cases. This includes asymptotic and non-asymptotic convergence results with natural and easily verifiable conditions, as well as explicit bounds on the convergence rates. Importantly, the theory is also general in that it can be applied to other intractable optimisation problems. A main novelty of the work is that the stochastic gradient estimates of our scheme are constructed from inexact proximal Markov chain Monte Carlo samplers. This allows the use of samplers that scale efficiently to large problems and for which we have precise theoretical guarantees.