2019/09/06 by Stefano Massaroli, Michael Poli, Massaroli, Stefano +11 · 1 citation
Biochemistry, Genetics and Molecular Biology · Engineering · Physics and Astronomy · #Advanced Electron Microscopy Techniques and Applications #Control and Stability of Dynamical Systems #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Model Reduction and Neural Networks #Neural and Evolutionary Computing (cs.NE) #Systems and Control (eess.SY) #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.1909.02702
openalex publication_date 2019/09/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Neural networks are discrete entities: subdivided into discrete layers and parametrized by weights which are iteratively optimized via difference equations. Recent work proposes networks with layer outputs which are no longer quantized but are solutions of an ordinary differential equation (ODE); however, these networks are still optimized via discrete methods (e.g. gradient descent). In this paper, we explore a different direction: namely, we propose a novel framework for learning in which the parameters themselves are solutions of ODEs. By viewing the optimization process as the evolution of a port-Hamiltonian system, we can ensure convergence to a minimum of the objective function. Numerical experiments have been performed to show the validity and effectiveness of the proposed methods.