Depth separation for reduced deep networks in nonlinear model reduction: Distilling shock waves in nonlinear hyperbolic problems

2020/07/28 by Donsub Rim, Rim, Donsub, Luca Venturi +5
Decision Sciences · Engineering · Physics and Astronomy · #41A46 #65M22 #68T07 #Elasticity and Material Modeling #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Numerical Analysis (math.NA) #Probabilistic and Robust Engineering Design

paper · pdf · doi:10.48550/arxiv.2007.13977

openalex publication_date 2020/07/28 · openalex created_date 2020/08/03 · openalex updated_date 2026/07/28

Abstract

Classical reduced models are low-rank approximations using a fixed basis designed to achieve dimensionality reduction of large-scale systems. In this work, we introduce reduced deep networks, a generalization of classical reduced models formulated as deep neural networks. We prove depth separation results showing that reduced deep networks approximate solutions of parametrized hyperbolic partial differential equations with approximation error ε with O(|log(ε)|) degrees of freedom, even in the nonlinear setting where solutions exhibit shock waves. We also show that classical reduced models achieve exponentially worse approximation rates by establishing lower bounds on the relevant Kolmogorov N-widths.

Citations

• Tensorizing Neural Networks
• Distilling the Knowledge in a Neural Network
• A Survey of Projection-Based Model Reduction Methods for Parametric Dynamical Systems
• A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks
• Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
• Manifold Approximations via Transported Subspaces: Model reduction for transport-dominated problems

Cited by

• Reconstruction of finite volume solution for parameter-dependent linear hyperbolic conservation laws

Related

• Capturing Shock Waves by Relaxation Neural Networks
• Deep-HyROMnet: A deep learning-based operator approximation for hyper-reduction of nonlinear parametrized PDEs
• Deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear partial differential equations
• A Theoretical Analysis of Deep Neural Networks and Parametric PDEs
• Reduced Basis Approximations of Parameterized Dynamical Partial Differential Equations via Neural Networks