2023/09/14 by Yuesheng Xu, Xu, Yuesheng, Taishan Zeng +1 · 1 citation
Physics and Astronomy · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #FOS: Mathematics #Model Reduction and Neural Networks #Numerical Analysis (math.NA)
paper · pdf · doi:10.48550/arxiv.2309.07401
openalex publication_date 2023/09/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Deep neural networks (DNNs) show great promise for solving partial differential equations (PDEs), but their deep architectures introduce complex, large-scale, non-convex optimization challenges. Nonlinear PDEs, like the viscous Burgers' equation, compound these difficulties due to steep gradients and shock-like solutions. To address this, we propose a two-stage multi-grade deep learning (TS-MGDL) method. In the first stage, shallow networks are trained progressively grade by grade to fit the target function from low- to high-frequency components; previously learned grades are frozen, and each new residual block is trained solely to minimize the remaining approximation error. The second stage unfreezes and retrains selected layers using the first-stage network as initialization, achieving an interpretable, stable hierarchical refinement while mitigating optimization complexity. Furthermore, we theoretically prove that each grade and stage in TS-MGDL monotonically reduces the loss function under an appropriate optimization strategy. Numerical experiments on 1D, 2D, and 3D viscous Burgers' equations demonstrate that TS-MGDL significantly outperforms single-grade learning (SGL), reducing predictive errors by up to a factor of 60.