2022/04/27 by Pietro Vertechi, Vertechi, Pietro, Mattia G. Bergomi +1
Computer Science · Physics and Astronomy · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning and Algorithms #Model Reduction and Neural Networks #Neural Networks and Applications #Optimization and Control (math.OC)
paper · pdf · doi:10.48550/arxiv.2204.12786
openalex publication_date 2022/04/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We provide a unifying framework where artificial neural networks and their architectures can be formally described as particular cases of a general mathematical construction--machines of finite depth. Unlike neural networks, machines have a precise definition, from which several properties follow naturally. Machines of finite depth are modular (they can be combined), efficiently computable and differentiable. The backward pass of a machine is again a machine and can be computed without overhead using the same procedure as the forward pass. We prove this statement theoretically and practically, via a unified implementation that generalizes several classical architectures--dense, convolutional, and recurrent neural networks with a rich shortcut structure--and their respective backpropagation rules.