Uniform Convergence with Square-Root Lipschitz Loss

2023/06/22 by Lijia Zhou, Zhen Dai, Zhou, Lijia +5
Engineering · Materials Science · Physics and Astronomy · #Advanced X-ray Imaging Techniques #Electron and X-Ray Spectroscopy Techniques #FOS: Computer and information sciences #Hydrocarbon exploration and reservoir analysis #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.2306.13188

openalex publication_date 2023/06/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We establish generic uniform convergence guarantees for Gaussian data in terms of the Rademacher complexity of the hypothesis class and the Lipschitz constant of the square root of the scalar loss function. We show how these guarantees substantially generalize previous results based on smoothness (Lipschitz constant of the derivative), and allow us to handle the broader class of square-root-Lipschitz losses, which includes also non-smooth loss functions appropriate for studying phase retrieval and ReLU regression, as well as rederive and better understand "optimistic rate" and interpolation learning guarantees.

Related

• On Lipschitz Continuity and Smoothness of Loss Functions in Learning to Rank
• Lipschitz Interpolation: Non-parametric Convergence under Bounded Stochastic Noise
• Time/Accuracy Tradeoffs for Learning a ReLU with respect to Gaussian Marginals
• Exploiting Smoothness in Statistical Learning, Sequential Prediction, and Stochastic Optimization
• LALR: Theoretical and Experimental validation of Lipschitz Adaptive\n Learning Rate in Regression and Neural Networks