Nonconvex Penalization in Sparse Estimation: An Approach Based on the Bernstein Function

2015/10/29 by Zhang, Zhihua
#FOS: Computer and information sciences #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.1510.08633

Abstract

In this paper we study nonconvex penalization using Bernstein functions whose first-order derivatives are completely monotone. The Bernstein function can induce a class of nonconvex penalty functions for high-dimensional sparse estimation problems. We derive a thresholding function based on the Bernstein penalty and discuss some important mathematical properties in sparsity modeling. We show that a coordinate descent algorithm is especially appropriate for regression problems penalized by the Bernstein function. We also consider the application of the Bernstein penalty in classification problems and devise a proximal alternating linearized minimization method. Based on theory of the Kurdyka-Lojasiewicz inequality, we conduct convergence analysis of these alternating iteration procedures. We particularly exemplify a family of Bernstein nonconvex penalties based on a generalized Gamma measure and conduct empirical analysis for this family.

Related

• The Bernstein Function: A Unifying Framework of Nonconvex Penalization in Sparse Estimation
• Optimization with Sparsity-Inducing Penalties
• Nonconvex flexible sparsity regularization: theory and monotone numerical schemes
• Computational approaches to non-convex, sparsity-inducing multi-penalty\n regularization
• Sparse Approximation via Penalty Decomposition Methods