2021/10/21 by Chanwoo Park, Ernest K. Ryu, Park, Chanwoo +1 · 1 citation
Computer Science · Engineering · #FOS: Mathematics #Machine Learning and Algorithms #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.2110.11035
openalex publication_date 2021/10/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In this work, we present a novel algorithm design methodology that finds the optimal algorithm as a function of inequalities. Specifically, we restrict convergence analyses of algorithms to use a prespecified subset of inequalities, rather than utilizing all true inequalities, and find the optimal algorithm subject to this restriction. This methodology allows us to design algorithms with certain desired characteristics. As concrete demonstrations of this methodology, we find new state-of-the-art accelerated first-order gradient methods using randomized coordinate updates and backtracking line searches.