Reversible and irreversible bracket-based dynamics for deep graph neural networks

2023/05/24 by Anthony Gruber, Kookjin Lee, Gruber, Anthony +3 · 7 citations
Engineering · Physics and Astronomy · #FOS: Computer and information sciences #Lattice Boltzmann Simulation Studies #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Quantum many-body systems

paper · pdf · doi:10.48550/arxiv.2305.15616

openalex publication_date 2023/05/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

Recent works have shown that physics-inspired architectures allow the training of deep graph neural networks (GNNs) without oversmoothing. The role of these physics is unclear, however, with successful examples of both reversible (e.g., Hamiltonian) and irreversible (e.g., diffusion) phenomena producing comparable results despite diametrically opposed mechanisms, and further complications arising due to empirical departures from mathematical theory. This work presents a series of novel GNN architectures based upon structure-preserving bracket-based dynamical systems, which are provably guaranteed to either conserve energy or generate positive dissipation with increasing depth. It is shown that the theoretically principled framework employed here allows for inherently explainable constructions, which contextualize departures from theory in current architectures and better elucidate the roles of reversibility and irreversibility in network performance.

Citations

• Adam: A Method for Stochastic Optimization
• Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
• Pitfalls of Graph Neural Network Evaluation
• Hamiltonian Neural Networks
• Continuous Graph Neural Networks
• Dissipative SymODEN: Encoding Hamiltonian Dynamics with Dissipation and Control into Deep Learning
• A Note on Over-Smoothing for Graph Neural Networks
• Enforcing exact physics in scientific machine learning: a data-driven exterior calculus on graphs
• Graph neural networks: A review of methods and applications
• Structure-preserving Sparse Identification of Nonlinear Dynamics for Data-driven Modeling
• Understanding and Mitigating Gradient Flow Pathologies in Physics-Informed Neural Networks
• When and why PINNs fail to train: A neural tangent kernel perspective
• Graph-Coupled Oscillator Networks
• Understanding convolution on graphs via energies
• Machine learning structure preserving brackets for forecasting irreversible processes
• Targeted Branching for the Maximum Independent Set Problem Using Graph Neural Networks
• Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints
• GREAD: Graph Neural Reaction-Diffusion Networks
• Exploiting Generative AI to Scale up Intelligent Tutoring Systems
• OpenAI Gym

Cited by

• Domain-Decomposed Graph Neural Network Surrogate Modeling for Ice Sheets
• Variational Neural Networks for Observable Thermodynamics (V-NOTS)
• Data-driven particle dynamics: Structure-preserving coarse-graining for emergent behavior in non-equilibrium systems
• Reduced-order modeling of Hamiltonian dynamics based on symplectic neural networks
• Meta-learning Structure-Preserving Dynamics
• Structure-Preserving Digital Twins via Conditional Neural Whitney Forms
• On the under-reaching phenomenon in message-passing neural PDE solvers: revisiting the CFL condition

Related

• Reversible Architectures for Arbitrarily Deep Residual Neural Networks
• Unravelling the Performance of Physics-informed Graph Neural Networks for Dynamical Systems
• Resilient Graph Neural Networks: A Coupled Dynamical Systems Approach
• Graph-Coupled Oscillator Networks
• Port-Hamiltonian Architectural Bias for Long-Range Propagation in Deep Graph Networks