A stochastic alternating direction method of multipliers for non-smooth and non-convex optimization

2020/12/14 by Fengmiao Bian, Jingwei Liang, Xiaoqun Zhang · 21 citations
Computer Science · Engineering · Mathematics · #Applied mathematics #Convex optimization #Geometry #Mathematical analysis #Mathematical optimization #Mathematics #Numerical methods in inverse problems #Regular polygon #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques #math.OC

paper · pdf · doi:10.1088/1361-6420/ac0966

published in Inverse Problems 37(7), 075009 (IOP Publishing)

arxiv created 2020/12/14 · openalex publication_date 2021/06/08 · openalex created_date 2021/06/22 · arxiv updated 2021/08/11 · openalex updated_date 2026/08/06

Abstract

Abstract Alternating direction method of multipliers (ADMM) is a popular first-order method owing to its simplicity and efficiency. However, similar to other proximal splitting methods, the performance of ADMM degrades significantly when the scale of optimization problems to solve becomes large. In this paper, we consider combining ADMM with a class of variance-reduced stochastic gradient estimators for solving large-scale non-convex and non-smooth optimization problems. Global convergence of the generated sequence is established under the additional assumption that the object function satisfies Kurdyka-Łojasiewicz property. Numerical experiments on graph-guided fused lasso and computed tomography are presented to demonstrate the performance of the proposed methods.

Citations

• Convergence of Bregman alternating direction method with multipliers for nonconvex composite problems
• Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods
• A Stochastic Approximation Method
• Proximal alternating linearized minimization for nonconvex and nonsmooth problems
• Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Łojasiewicz Inequality
• Mini-Batch Stochastic ADMMs for Nonconvex Nonsmooth Optimization
• Scalable Stochastic Alternating Direction Method of Multipliers
• Faster Stochastic Alternating Direction Method of Multipliers for Nonconvex Optimization
• SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
• Accelerated Variance Reduced Stochastic ADMM
• ℓ₀ Minimization for wavelet frame based image restoration
• SPRING: A fast stochastic proximal alternating method for non-smooth non-convex optimization
• An Accelerated Linearized Alternating Direction Method of Multipliers

Cited by

• Perturbed Proximal Gradient ADMM for Nonconvex Composite Optimization
• Stochastic momentum ADMM for nonconvex and nonsmooth optimization with application to PnP algorithm