2020/03/03 by Qianggong Zhang, Zhang, Qianggong, Yanyang Gu +8 · 5 citations
Computer Science · Engineering · #Advanced Neural Network Applications #Artificial intelligence #Artificial neural network #Backpropagation #Class (philosophy) #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Control engineering #Engineering #FOS: Computer and information sciences #Feed forward #Feedforward neural network #Function (biology) #Human Pose and Action Recognition #Layer (electronics) #Medical Image Segmentation Techniques #Relevance (law) #Theoretical computer science #cs.CV
paper · pdf · doi:10.48550/arxiv.2003.01822
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2020/03/03 · arxiv created 2020/06/03 · arxiv updated 2020/06/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06
In conventional formulations of multilayer feedforward neural networks, the individual layers are customarily defined by explicit functions. In this paper we demonstrate that defining individual layers in a neural network implicitly provide much richer representations over the standard explicit one, consequently enabling a vastly broader class of end-to-end trainable architectures. We present a general framework of implicitly defined layers, where much of the theoretical analysis of such layers can be addressed through the implicit function theorem. We also show how implicitly defined layers can be seamlessly incorporated into existing machine learning libraries. In particular with respect to current automatic differentiation techniques for use in backpropagation based training. Finally, we demonstrate the versatility and relevance of our proposed approach on a number of diverse example problems with promising results.