Origins and mitigation of double descent in reduced order modeling

2026/07/29 by Andrei A. Klishin, J. Nathan Kutz, Krithika Manohar
Computer Science · Mathematics · Physics and Astronomy · #cs.LG #math.DS #physics.data-an #stat.ML

paper · pdf

20 RevTeX pages, 11 figures

arxiv created 2026/07/29 · arxiv updated 2026/07/30

Abstract

Latent low-dimensional structure in datasets of natural and engineered systems enables their sparse sensing, or full-state reconstruction from historical data and very few carefully chosen localized measurements. Depending on the reconstruction algorithm, sensor locations, and measurement noise, the reconstruction risk curves demonstrate a diversity of patterns including a dramatic peak in error known as double descent in Machine Learning literature. Here we explore those scenarios under a unified Data-Noise Averaging theory. Qualitatively, we formulate sufficient criteria for double descent to emerge through a catastrophic amplification of a pathological signal in reconstruction. Quantitatively, we predict the detailed risk curves at a fraction of computational cost, trace reconstruction instability to individual sensors and their combinations, and provide regularization mechanisms to mitigate the instability. We demonstrate results for both static reconstruction of Sea Surface Temperature patterns and time integration of a reduced order model of a PDE.

Citations

• PySensors 2.0: A Python Package for Sparse Sensor Placement
• Bridging the Gap Between Deterministic and Probabilistic Approaches to State Estimation
• Reduced order modeling with shallow recurrent decoder networks
• State Estimation Using Sparse DEIM and Recurrent Neural Networks
• Unveiling Multiple Descents in Unsupervised Autoencoders
• Sparse Discrete Empirical Interpolation Method: State Estimation from Few Sensors
• Data-Induced Interactions of Sparse Sensors Using Statistical Physics
• Modern Bayesian Experimental Design
• Sensing with shallow recurrent decoder networks
• Learning Dynamic Graphs, Too Slow
• Exposure theory for learning complex networks with random walks
• Information geometry for multiparameter models: New perspectives on the origin of simplicity
• The Shape of Learning Curves: a Review
• PySensors 2.0: A Python Package for Sparse Sensor Placement
• Multi-fidelity sensor selection: Greedy algorithms to place cheap and\n expensive sensors with cost constraints
• Optimal Regularization Can Mitigate Double Descent
• Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime
• Deep Double Descent: Where Bigger Models and More Data Hurt
• The generalization error of random features regression: Precise asymptotics and double descent curve
• The Generalization Error of Random Features Regression: Precise Asymptotics and the Double Descent Curve
• Minimizers of the Empirical Risk and Risk Monotonicity
• Surprises in High-Dimensional Ridgeless Least Squares Interpolation
• The Peaking Phenomenon in Semi-supervised Learning
• A Review of Modern Computational Algorithms for Bayesian Optimal Design

Cited by

• PySensors 2.0: A Python Package for Sparse Sensor Placement

Related

• Understanding the Double Descent Phenomenon in Deep Learning
• How more data can hurt: Instability and regularization in next-generation reservoir computing
• Reduced order modeling with shallow recurrent decoder networks
• State Estimation -- The Role of Reduced Models
• CS-SHRED: Enhancing SHRED for Robust Recovery of Spatiotemporal Dynamics