2024/02/01 by Shakil Rafi, Rafi, Shakil, Joshua Lee Padgett +3
Computer Science · #62M45 #Combinatorics (math.CO) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE) #Numerical Analysis (math.NA)
paper · pdf · doi:10.48550/arxiv.2402.01058
openalex publication_date 2024/02/01 · openalex created_date 2024/02/06 · openalex updated_date 2026/07/28
We make the case for neural network objects and extend an already existing neural network calculus explained in detail in Chapter 2 on \citebigbook. Our aim will be to show that, yes, indeed, it makes sense to talk about neural network polynomials, neural network exponentials, sine, and cosines in the sense that they do indeed approximate their real number counterparts subject to limitations on certain of their parameters, q, and ε. While doing this, we show that the parameter and depth growth are only polynomial on their desired accuracy (defined as a 1-norm difference over ℝ), thereby showing that this approach to approximating, where a neural network in some sense has the structural properties of the function it is approximating is not entire intractable.