2023/09/12 by Jingpu Cheng, Cheng, Jingpu, Qianxiao Li +5 · 4 citations
Computer Science · Physics and Astronomy · #41A05 #68T07 #93B05 #Adversarial Robustness in Machine Learning #Dynamical Systems (math.DS) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Neural Networks and Applications #Optimization and Control (math.OC)
paper · pdf · doi:10.48550/arxiv.2309.06015
openalex publication_date 2023/09/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We investigate the expressive power of deep residual neural networks idealized as continuous dynamical systems through control theory. Specifically, we consider two properties that arise from supervised learning, namely universal interpolation - the ability to match arbitrary input and target training samples - and the closely related notion of universal approximation - the ability to approximate input-target functional relationships via flow maps. Under the assumption of affine invariance of the control family, we give a characterisation of universal interpolation, showing that it holds for essentially any architecture with non-linearity. Furthermore, we elucidate the relationship between universal interpolation and universal approximation in the context of general control systems, showing that the two properties cannot be deduced from each other. At the same time, we identify conditions on the control family and the target function that ensures the equivalence of the two notions.