Nonasymptotic estimates for Stochastic Gradient Langevin Dynamics under local conditions in nonconvex optimization

2019/10/04 by Ying Zhang, Ömer Deniz Akyıldız, Zhang, Ying +5 · 9 citations
Engineering · Mathematics · #60J20 #60J22 #62D05 #65C05 #65C40 #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Markov Chains and Monte Carlo Methods #Point processes and geometric inequalities #Probability (math.PR) #Sparse and Compressive Sensing Techniques #Statistics Theory (math.ST)

paper · pdf · doi:10.48550/arxiv.1910.02008

openalex publication_date 2019/10/04 · openalex created_date 2019/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we are concerned with a non-asymptotic analysis of sampling algorithms used in nonconvex optimization. In particular, we obtain non-asymptotic estimates in Wasserstein-1 and Wasserstein-2 distances for a popular class of algorithms called Stochastic Gradient Langevin Dynamics (SGLD). In addition, the aforementioned Wasserstein-2 convergence result can be applied to establish a non-asymptotic error bound for the expected excess risk. Crucially, these results are obtained under a local Lipschitz condition and a local dissipativity condition where we remove the uniform dependence in the data stream. We illustrate the importance of this relaxation by presenting examples from variational inference and from index tracking optimization.

Citations

• Global Convergence of Langevin Dynamics Based Algorithms for Nonconvex Optimization
• Quantitative Harris type theorems for diffusions and McKean-Vlasov\n processes

Cited by

• Learning Latent Energy-Based Models via Interacting Particle Langevin Dynamics
• Flatness-Aware Stochastic Gradient Langevin Dynamics
• Uniform-in-time convergence bounds for Persistent Contrastive Divergence Algorithms
• Anchored Langevin Algorithms
• Contractive kinetic Langevin samplers beyond global Lipschitz continuity
• Regime-Switching Langevin Monte Carlo Algorithms
• Modelling birdsong transmission with methods from molecular sequence analysis
• Non-asymptotic estimates for TUSLA algorithm for non-convex learning with applications to neural networks with ReLU activation function
• From Global to Local: A Scalable Benchmark for Local Posterior Sampling

Related

• On stochastic gradient Langevin dynamics with dependent data streams: the fully non-convex case
• Non-convex learning via Stochastic Gradient Langevin Dynamics: a\n nonasymptotic analysis
• A Hitting Time Analysis of Stochastic Gradient Langevin Dynamics
• Nonasymptotic analysis of Stochastic Gradient Hamiltonian Monte Carlo under local conditions for nonconvex optimization
• Non-Asymptotic Guarantees For Sampling by Stochastic Gradient Descent