Finite Sample Identification of Wide Shallow Neural Networks with Biases

2022/11/08 by Massimo Fornasier, Fornasier, Massimo, Timo Klock +5 · 1 citation
Computer Science · Physics and Astronomy · #65D15 #68T07 #90C26 #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Model Reduction and Neural Networks #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2211.04589

openalex publication_date 2022/11/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Artificial neural networks are functions depending on a finite number of parameters typically encoded as weights and biases. The identification of the parameters of the network from finite samples of input-output pairs is often referred to as the teacher-student model, and this model has represented a popular framework for understanding training and generalization. Even if the problem is NP-complete in the worst case, a rapidly growing literature -- after adding suitable distributional assumptions -- has established finite sample identification of two-layer networks with a number of neurons m=\mathcal O(D), D being the input dimension. For the range D

Cited by

• Data Augmentation Techniques to Reverse-Engineer Neural Network Weights from Input-Output Queries

Related

• Finite Samples for Shallow Neural Networks
• A function space analysis of finite neural networks with insights from sampling theory
• Generalization performance of narrow one-hidden layer networks in the teacher-student setting
• Robust and Resource Efficient Identification of Shallow Neural Networks\n by Fewest Samples
• Sampling weights of deep neural networks