Automatic and Structure-Aware Sparsification of Hybrid Neural ODEs

2025/05/25 by Bob Junyi Zou, Tian Lu, Zou, Bob Junyi +1
Engineering · #Advanced Measurement and Metrology Techniques #Advanced Numerical Analysis Techniques #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Tribology and Lubrication Engineering

paper · pdf · doi:10.48550/arxiv.2505.18996

openalex publication_date 2025/05/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Hybrid neural ordinary differential equations (neural ODEs) integrate mechanistic models with neural ODEs, offering strong inductive bias and flexibility, and are particularly advantageous in data-scarce healthcare settings. However, excessive latent states and interactions from mechanistic models can lead to training inefficiency and over-fitting, limiting practical effectiveness of hybrid neural ODEs. In response, we propose a new hybrid pipeline for automatic state selection and structure optimization in mechanistic neural ODEs, combining domain-informed graph modifications with data-driven regularization to sparsify the model for improving predictive performance and stability while retaining mechanistic plausibility. Experiments on synthetic and real-world data show improved predictive performance and robustness with desired sparsity, establishing an effective solution for hybrid model reduction in healthcare applications.

Citations

• The Contextual Lasso: Sparse Linear Models via Deep Neural Networks
• On the Parameterization and Initialization of Diagonal State Space Models
• On Neural Differential Equations
• The Causal-Neural Connection: Expressiveness, Learnability, and Inference
• Integrating Expert ODEs into Neural ODEs: Pharmacology and Disease Progression
• Physics-informed machine learning
• Stiff neural ordinary differential equations
• Neural Pharmacodynamic State Space Modeling
• Sparsity in Deep Learning: Pruning and growth for efficient inference\n and training in neural networks
• Neural Ordinary Differential Equations for Intervention Modeling
• Learning Insulin-Glucose Dynamics in the Wild
• Graph Structure of Neural Networks
• Learning to Simulate Complex Physics with Graph Networks
• Universal Differential Equations for Scientific Machine Learning
• Graph Neural Ordinary Differential Equations
• Learning Robust Representations with Graph Denoising Policy Network
• The Generalization-Stability Tradeoff In Neural Network Pruning
• The State of Sparsity in Deep Neural Networks
• Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
• Techniques for Interpretable Machine Learning
• Neural Ordinary Differential Equations
• Stable architectures for deep neural networks
• Temporal Convolutional Networks: A Unified Approach to Action\n Segmentation
• Learning Structured Sparsity in Deep Neural Networks
• Graph Sparsification by Effective Resistances
• A quantitative description of membrane current and its application to conduction and excitation in nerve
• Best Subset, Forward Stepwise or Lasso? Analysis and Recommendations Based on Extensive Comparisons

Related

• Meta-Solver for Neural Ordinary Differential Equations
• How to train your neural ODE: the world of Jacobian and kinetic regularization
• Mixed Precision Training of Neural ODEs
• Learning the Simplest Neural ODE
• Towards Understanding Normalization in Neural ODEs