A Generalization Bound for Nearly-Linear Networks

2024/07/09 by Eugène Golikov, Golikov, Eugene
Engineering · Mathematics · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Graph theory and applications #Machine Learning (cs.LG) #Machine Learning (stat.ML) #graph theory and CDMA systems

paper · pdf · doi:10.48550/arxiv.2407.06765

openalex publication_date 2024/07/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We consider nonlinear networks as perturbations of linear ones. Based on this approach, we present novel generalization bounds that become non-vacuous for networks that are close to being linear. The main advantage over the previous works which propose non-vacuous generalization bounds is that our bounds are a-priori: performing the actual training is not required for evaluating the bounds. To the best of our knowledge, they are the first non-vacuous generalization bounds for neural nets possessing this property.

Related

• Lower Bounds on the Generalization Error of Nonlinear Learning Models
• On Generalization Bounds for Neural Networks with Low Rank Layers
• On Tighter Generalization Bound for Deep Neural Networks: CNNs, ResNets, and Beyond
• Fantastic Generalization Measures are Nowhere to be Found
• Algorithm-Dependent Generalization Bounds for Overparameterized Deep Residual Networks