Sparse-Input Neural Network using Group Concave Regularization

2023/07/01 by Bin Luo, Luo, Bin, Susan Halabi +1
Computer Science · Engineering · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and ELM #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques

paper · doi:10.48550/arxiv.2307.00344

openalex publication_date 2023/07/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

norm of weights from all outgoing connections of each input node, and thus obtain a neural net that only uses a small subset of the original variables. In addition, we develop an effective algorithm based on backward path-wise optimization to yield stable solution paths, in order to tackle the challenge of complex optimization landscapes. We provide a rigorous theoretical analysis of the proposed framework, establishing finite-sample guarantees for both variable selection consistency and prediction accuracy. These results are supported by extensive simulation studies and real data applications, which demonstrate the finite-sample performance of the estimator in feature selection and prediction across continuous, binary, and time-to-event outcomes.

Related

• Lockout: Sparse Regularization of Neural Networks
• Sparse-Input Neural Networks for High-dimensional Nonparametric Regression and Classification
• Transformed ℓ₁ Regularization for Learning Sparse Deep Neural Networks
• Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel
• Regularization for Unsupervised Deep Neural Nets