Accelerated Block Coordinate Proximal Gradients with Applications in\n High Dimensional Statistics

2017/10/15 by Tsz Kit Lau, Yuan Yao, Lau, Tsz Kit +1
Computer Science · Engineering · #Distributed Sensor Networks and Detection Algorithms #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Photoacoustic and Ultrasonic Imaging #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.1710.05338

openalex publication_date 2017/10/15 · openalex created_date 2022/09/03 · openalex updated_date 2026/07/28

Abstract

Nonconvex optimization problems arise in different research fields and arouse\nlots of attention in signal processing, statistics and machine learning. In\nthis work, we explore the accelerated proximal gradient method and some of its\nvariants which have been shown to converge under nonconvex context recently. We\nshow that a novel variant proposed here, which exploits adaptive momentum and\nblock coordinate update with specific update rules, further improves the\nperformance of a broad class of nonconvex problems. In applications to sparse\nlinear regression with regularizations like Lasso, grouped Lasso, capped\n\ℓ₁ and SCAP, the proposed scheme enjoys provable local linear\nconvergence, with experimental justification.\n

Citations

• Nearly unbiased variable selection under minimax concave penalty
• Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods
• Proximal alternating linearized minimization for nonconvex and nonsmooth problems
• Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Łojasiewicz Inequality
• A General Iterative Shrinkage and Thresholding Algorithm for Non-convex Regularized Optimization Problems

Related

• Convergence Analysis of Proximal Gradient with Momentum for Nonconvex Optimization
• An Accelerated Block Proximal Framework with Adaptive Momentum for Nonconvex and Nonsmooth Optimization
• Smoothing Proximal Gradient Methods for Nonsmooth Sparsity Constrained Optimization: Optimality Conditions and Global Convergence
• Accelerated Proximal Stochastic Dual Coordinate Ascent for Regularized Loss Minimization
• An Accelerated Proximal Coordinate Gradient Method and its Application to Regularized Empirical Risk Minimization