Robustly Learning Monotone Generalized Linear Models via Data Augmentation

2025/02/12 by Nikos Zarifis, Puqian Wang, Zarifis, Nikos +5 · 1 citation
Computer Science · #Face and Expression Recognition #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.2502.08611

Abstract

We study the task of learning Generalized Linear models (GLMs) in the agnostic model under the Gaussian distribution. We give the first polynomial-time algorithm that achieves a constant-factor approximation for any monotone Lipschitz activation. Prior constant-factor GLM learners succeed for a substantially smaller class of activations. Our work resolves a well-known open problem, by developing a robust counterpart to the classical GLMtron algorithm (Kakade et al., 2011). Our robust learner applies more generally, encompassing all monotone activations with bounded (2+ζ)-moments, for any fixed ζ>0 -- a condition that is essentially necessary. To obtain our results, we leverage a novel data augmentation technique with decreasing Gaussian noise injection and prove a number of structural results that may be useful in other settings.

Cited by

• Robustly Learning Monotone Single-Index Models

Related

• Robustly Learning Monotone Single-Index Models
• Agnostic Proper Learning of Halfspaces under Gaussian Marginals
• Robustly Learning Single-Index Models via Alignment Sharpness
• Robust Model Selection and Nearly-Proper Learning for GMMs
• The Optimality of Polynomial Regression for Agnostic Learning under Gaussian Marginals