2022/06/01 by Yi Guo, Zhaocheng Liu, Guo, Yi +15 · 2 citations
Engineering · Materials Science · Social Sciences · #Advanced Computing and Algorithms #FOS: Computer and information sciences #Industrial Vision Systems and Defect Detection #Information Retrieval (cs.IR) #Machine Learning in Materials Science
paper · pdf · doi:10.48550/arxiv.2206.00267
openalex publication_date 2022/06/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In industry, feature selection is a standard but necessary step to search for an optimal set of informative feature fields for efficient and effective training of deep Click-Through Rate (CTR) models. Most previous works measure the importance of feature fields by using their corresponding continuous weights from the model, then remove the feature fields with small weight values. However, removing many features that correspond to small but not exact zero weights will inevitably hurt model performance and not be friendly to hot-start model training. There is also no theoretical guarantee that the magnitude of weights can represent the importance, thus possibly leading to sub-optimal results if using these methods. To tackle this problem, we propose a novel Learnable Polarizing Feature Selection (LPFS) method using a smoothed-ℓ0 function in literature. Furthermore, we extend LPFS to LPFS++ by our newly designed smoothed-ℓ0-liked function to select a more informative subset of features. LPFS and LPFS++ can be used as gates inserted at the input of the deep network to control the active and inactive state of each feature. When training is finished, some gates are exact zero, while others are around one, which is particularly favored by the practical hot-start training in the industry, due to no damage to the model performance before and after removing the features corresponding to exact-zero gates. Experiments show that our methods outperform others by a clear margin, and have achieved great A/B test results in KuaiShou Technology.