Comparison of neural network training strategies for the simulation of dynamical systems

2025/12/03 by Paul Strasser, Strasser, Paul, Pfeffer, Andreas +6
Computer Science · Engineering · Physics and Astronomy · #Control Systems and Identification #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Model Reduction and Neural Networks

paper · pdf · doi:10.48550/arxiv.2512.03851

openalex publication_date 2025/12/03 · openalex created_date 2025/12/05 · openalex updated_date 2026/07/28

Abstract

Neural networks have become a widely adopted tool for modeling nonlinear dynamical systems from data. However, the choice of training strategy remains a key design decision, particularly for simulation tasks. This paper compares two predominant strategies: parallel and series-parallel training. The conducted empirical analysis spans five neural network architectures and two examples: a pneumatic valve test bench and an industrial robot benchmark. The study reveals that, even though series-parallel training dominates current practice, parallel training consistently yields better long-term prediction accuracy. Additionally, this work clarifies the often inconsistent terminology in the literature and relate both strategies to concepts from system identification. The findings suggest that parallel training should be considered the default training strategy for neural network-based simulation of dynamical systems.

Citations

• Learned Perceptive Forward Dynamics Model for Safe and Platform-aware Robotic Navigation
• Data-driven identification of nonlinear dynamical systems with LSTM autoencoders and Normalizing Flows
• Distributionally Robust Policy Learning under Concept Drifts
• Deep networks for system identification: a Survey
• Constructing Neural Network-Based Models for Simulating Dynamical Systems
• Tune: A Research Platform for Distributed Model Selection and Training
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• Capacity and Trainability in Recurrent Neural Networks
• Deep Residual Learning for Image Recognition
• Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
• Adam: A Method for Stochastic Optimization
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
• Long Short-Term Memory

Related

• Properties of neural networks with applications to modelling non-linear dynamical systems
• dynoNet: a neural network architecture for learning dynamical systems
• On feedforward control using physics-guided neural networks: Training cost regularization and optimized initialization
• Learning neural state-space models: do we need a state estimator?
• Constrained Block Nonlinear Neural Dynamical Models