An Iteratively Re-weighted Method for Problems with Sparsity-Inducing Norms

2019/07/02 by Feiping Nie, Nie, Feiping, Zhanxuan Hu +9
Decision Sciences · Engineering · Mathematics · #Advanced Optimization Algorithms Research #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Probabilistic and Robust Engineering Design #Sparse and Compressive Sensing Techniques

paper · pdf · doi:10.48550/arxiv.1907.01121

openalex publication_date 2019/07/02 · openalex created_date 2019/07/12 · openalex updated_date 2026/07/28

Abstract

This work aims at solving the problems with intractable sparsity-inducing norms that are often encountered in various machine learning tasks, such as multi-task learning, subspace clustering, feature selection, robust principal component analysis, and so on. Specifically, an Iteratively Re-Weighted method (IRW) with solid convergence guarantee is provided. We investigate its convergence speed via numerous experiments on real data. Furthermore, in order to validate the practicality of IRW, we use it to solve a concrete robust feature selection model with complicated objective function. The experimental results show that the model coupled with proposed optimization method outperforms alternative methods significantly.

Citations

• Nearly unbiased variable selection under minimax concave penalty
• Smooth minimization of non-smooth functions

Related

• Direct l_(2,p)-Norm Learning for Feature Selection
• Safe Screening Rules for Generalized Double Sparsity Learning
• Learning Cluster Structured Sparsity by Reweighting
• On Sparsity Inducing Regularization Methods for Machine Learning
• Enhancing Unsupervised Feature Selection via Double Sparsity Constrained Optimization