Regularising NARX models with multi-task learning

2025/01/08 by Sarah Bee, Bee, Sarah, Lawrence A. Bull +5
Computer Science · Engineering · #FOS: Computer and information sciences #Fault Detection and Control Systems #Machine Learning (cs.LG) #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.2501.04470

openalex publication_date 2025/01/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

A Nonlinear Auto-Regressive with eXogenous inputs (NARX) model can be used to describe time-varying processes; where the output depends on both previous outputs and current/previous external input variables. One limitation of NARX models is their propensity to overfit and result in poor generalisation for future predictions. The proposed method to help to overcome the issue of overfitting is a NARX model which predicts outputs at both the current time and several lead times into the future. This is a form of multi-task learner (MTL); whereby the lead time outputs will regularise the current time output. This work shows that for high noise level, MTL can be used to regularise NARX with a lower Normalised Mean Square Error (NMSE) compared to the NMSE of the independent learner counterpart.

Related

• NARX Identification using Derivative-Based Regularized Neural Networks
• Deep Energy-Based NARX Models
• Decoupling P-NARX models using filtered CPD
• Analysing Multi-Task Regression via Random Matrix Theory with Application to Time Series Forecasting
• A Multi-step Loss Function for Robust Learning of the Dynamics in Model-based Reinforcement Learning