2018/06/13 by Xingguo Li, Junwei Lu, Li, Xingguo +7 · 2 citations
Computer Science · Engineering · Mathematics · #FOS: Computer and information sciences #Face and Expression Recognition #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications #Sparse and Compressive Sensing Techniques #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.1806.05159
openalex publication_date 2018/06/13 · arxiv created 2019/07/03 · arxiv updated 2019/07/05 · openalex created_date 2022/10/02 · openalex updated_date 2026/07/28
We establish a margin based data dependent generalization error bound for a general family of deep neural networks in terms of the depth and width, as well as the Jacobian of the networks. Through introducing a new characterization of the Lipschitz properties of neural network family, we achieve significantly tighter generalization bounds than existing results. Moreover, we show that the generalization bound can be further improved for bounded losses. Aside from the general feedforward deep neural networks, our results can be applied to derive new bounds for popular architectures, including convolutional neural networks (CNNs) and residual networks (ResNets). When achieving same generalization errors with previous arts, our bounds allow for the choice of larger parameter spaces of weight matrices, inducing potentially stronger expressive ability for neural networks. Numerical evaluation is also provided to support our theory.