Backpropagation in unstable diffusions

2025/07/29 by Ni, Angxiu · 1 citation
#Computational Physics (physics.comp-ph) #Dynamical Systems (math.DS) #FOS: Mathematics #FOS: Physical sciences #Probability (math.PR)

paper · doi:10.48550/arxiv.2507.21497

Abstract

We derive the adjoint path-kernel method for the parameter-gradient of SDEs, where the observable is averaged at a particular time or over the stationary measure. Its cost is almost independent of the number of parameters; it extends the conventional backpropagation method to cases with gradient explosion. It works for non-hyperbolic systems with multiplicative noise controlled by parameters. We derive a Monte-Carlo-type algorithm and demonstrate it on the 40-dimensional Lorenz 96 system.

Citations

• Divergence-Kernel method for scores of random systems
• Malliavin Calculus for Score-based Diffusion Models
• Path-Kernel Method for Differentiating Unstable Diffusions
• Optimal Response for Hyperbolic Systems by the fast adjoint response method
• Ergodic and foliated kernel-differentiation method for linear responses of random systems
• Numerical Solution for Nonlinear 4D Variational Data Assimilation (4D-Var) via ADMM
• Backpropagation in hyperbolic chaos via adjoint shadowing
• On convergence of linear response formulae in some piecewise hyperbolic maps
• Non-hyperbolicity at large scales of a high-dimensional chaotic system
• Linear Range in Gradient Descent
• Inverse Problems and Data Assimilation
• Bismut Formula for Lions Derivative of Distribution Dependent SDEs and Applications
• Linear Response for dynamical systems with additive noise
• Linear response, or else
• (Non)uniqueness of critical points in variational data assimilation
• On the difficulty of training Recurrent Neural Networks
• Divergence-kernel method for linear responses of densities and generative models

Cited by

• Divergence-kernel method for linear responses of densities and generative models

Related

• Path-Kernel Method for Differentiating Unstable Diffusions
• AdjointDEIS: Efficient Gradients for Diffusion Models
• AdjointDPM: Adjoint Sensitivity Method for Gradient Backpropagation of Diffusion Probabilistic Models
• Divergence-kernel method for linear responses of densities and generative models
• A Stochastic Gradient Descent Theorem and the Back-Propagation Algorithm