2013/11/15 by Ziqiang Shi, Shi, Ziqiang, Rujie Liu +1
Computer Science · Decision Sciences · Engineering · #Advanced Bandit Algorithms Research #FOS: Mathematics #Numerical Analysis (math.NA) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.1311.3832
openalex publication_date 2013/11/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Online and stochastic gradient methods have emerged as potent tools in large scale optimization with both smooth convex and nonsmooth convex problems from the classes C1,1(\realsp) and C1,0(\realsp) respectively. However to our best knowledge, there is few paper to use incremental gradient methods to optimization the intermediate classes of convex problems with Hölder continuous functions C1,v(\realsp). In order fill the difference and gap between methods for smooth and nonsmooth problems, in this work, we propose the several online and stochastic universal gradient methods, that we do not need to know the actual degree of smoothness of the objective function in advance. We expanded the scope of the problems involved in machine learning to Hölder continuous functions and to propose a general family of first-order methods. Regret and convergent analysis shows that our methods enjoy strong theoretical guarantees. For the first time, we establish an algorithms that enjoys a linear convergence rate for convex functions that have Hölder continuous gradients.