2025/06/29 by Heinz H. Bauschke, Bauschke, Heinz H., Tran Thanh Tung +1
Computer Science · Decision Sciences · Economics, Econometrics and Finance · #65K05 #68W20 #68W40 #90C25 #FOS: Mathematics #Functional Analysis (math.FA) #Optimization and Control (math.OC) #Primary 90C15 #Risk and Portfolio Optimization #Secondary 68T07 #Stochastic Gradient Optimization Techniques #Stochastic processes and financial applications
paper · pdf · doi:10.48550/arxiv.2506.23303
openalex publication_date 2025/06/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Stochastic Gradient Descent (SGD) with Polyak's stepsize has recently gained renewed attention in stochastic optimization. Recently, Orvieto, Lacoste-Julien, and Loizou introduced a decreasing variant of Polyak's stepsize, where convergence relies on a boundedness assumption of the iterates. They established that this assumption holds under strong convexity. In this paper, we extend their result by proving that boundedness also holds for a broader class of objective functions, including coercive functions. We also present a case in which boundedness may or may not hold.