Convergence to minima for the continuous version of Backtracking Gradient Descent

2019/11/11 by Truong, Tuyen Trung
#FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Numerical Analysis (math.NA) #Optimization and Control (math.OC)

paper · doi:10.48550/arxiv.1911.04221

Abstract

The main result of this paper is: \bf Theorem. Let f:ℝ^k→ ℝ be a C¹ function, so that ∇ f is locally Lipschitz continuous. Assume moreover that f is C² near its generalised saddle points. Fix real numbers δ₀>0 and 0

Related

• A nonsmooth nonconvex descent algorithm
• Gradient descent avoids strict saddles with a simple line-search method too
• Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regions
• Fast convergence to non-isolated minima: four equivalent conditions for C² functions
• Gradient Descent in the Absence of Global Lipschitz Continuity of the Gradients