2016/12/29 by Sen Na, Na, Sen, Cho‐Jui Hsieh +1 · 1 citation
Computer Science · Engineering · Mathematics · #Advanced MIMO Systems Optimization #Computation (stat.CO) #Direction-of-Arrival Estimation Techniques #FOS: Computer and information sciences #FOS: Mathematics #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Statistical Methods and Inference
paper · pdf · doi:10.48550/arxiv.1612.09357
openalex publication_date 2016/12/29 · openalex created_date 2022/10/03 · openalex updated_date 2026/07/28
Minimizing sum of two functions under a linear constraint is what we called\nsplitting problem. This convex optimization has wide applications in machine\nlearning problems, such as Lasso, Group Lasso and Sparse logistic regression. A\nrecent paper by Gu et al (2015) developed a Semi-Proximal-Based Strictly\nContractive Peaceman-Rachford Splitting Method (SPB-SPRSM), which is an\nextension of Strictly Contractive Peaceman-Rachford Splitting Method (SPRSM)\nproposed by He et al (2014). By introducing semi-proximal terms and using two\ndifferent relaxation factors, SPB-SPRSM showed a more flexiable applicability\ncomparing to its origin SPRSM and widely-used Alternating Direction Method of\nMultipliers (ADMM) algorithm, although all of them have O(1/t) convergence\nrate. In this paper, we develop a stochastic version of SPB-SPRSM algorithm,\nwhere only a subset of samples (even one sample) are used at each iteration.\nThe resulting algorithm, Stochastic SPB-SPRSM, is more flexiable than\nStochastic ADMM and other ADMM-based algorithms on both simulations and real\ndatasets. Moreover, we prove O(1/\√(t)) convergence rate in ergodic sense,\nwhich is the same with Stochastic ADMM algorithm under the same assumption. But\nas shown in He et al (2014) that SPRSM based algorithms will always converge\nfaster than ADMM in apllication, our proposed algorithm will also preserve this\nadvantage.\n