Stochastic Weakly Convex Optimization Beyond Lipschitz Continuity

2024/01/25 by Gao, Wenzhi, Deng, Qi · 4 citations
#FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Optimization and Control (math.OC)

paper · doi:10.48550/arxiv.2401.13971

Abstract

This paper considers stochastic weakly convex optimization without the standard Lipschitz continuity assumption. Based on new adaptive regularization (stepsize) strategies, we show that a wide class of stochastic algorithms, including the stochastic subgradient method, preserve the O ( 1 / √(K)) convergence rate with constant failure rate. Our analyses rest on rather weak assumptions: the Lipschitz parameter can be either bounded by a general growth function of ‖x‖ or locally estimated through independent random samples.

Cited by

• A Smooth Approximation Framework for Weakly Convex Optimization
• Quantitative Convergence Analysis of Projected Stochastic Gradient Descent for Non-Convex Losses via the Goldstein Subdifferential
• Revisiting Randomized Smoothing: Nonsmooth Nonconvex Optimization Beyond Global Lipschitz Continuity
• Stochastic Weakly Convex Optimization Under Heavy-Tailed Noises

Related

• Convergence Rates for Deterministic and Stochastic Subgradient Methods Without Lipschitz Continuity
• Beyond Convexity: Stochastic Quasi-Convex Optimization
• Graphical Convergence of Subgradients in Nonconvex Optimization and Learning
• Benign Underfitting of Stochastic Gradient Descent
• Adaptive First-and Zeroth-order Methods for Weakly Convex Stochastic Optimization Problems