Neural Ordinary Differential Equations

2018/06/19 by Ricky T. Q. Chen, Chen, Ricky T. Q., Yulia Rubanova +5 · 5 voices · 560 citations
Physics and Astronomy · Computer Science · #Model Reduction and Neural Networks #Generative Adversarial Networks and Image Synthesis #Computational Physics and Python Applications

paper · pdf · doi:10.48550/arxiv.1806.07366

Abstract

We introduce a new family of deep neural network models. Instead of specifying a discrete sequence of hidden layers, we parameterize the derivative of the hidden state using a neural network. The output of the network is computed using a black-box differential equation solver. These continuous-depth models have constant memory cost, adapt their evaluation strategy to each input, and can explicitly trade numerical precision for speed. We demonstrate these properties in continuous-depth residual networks and continuous-time latent variable models. We also construct continuous normalizing flows, a generative model that can train by maximum likelihood, without partitioning or ordering the data dimensions. For training, we show how to scalably backpropagate through any ODE solver, without access to its internal operations. This allows end-to-end training of ODEs within larger models.

Citations

• Auto-Encoding Variational Bayes
• Gradient-based learning applied to document recognition

Cited by

• SESaMo: Symmetry-Enforcing Stochastic Modulation for Normalizing Flows
• Learning Transfers: Kan Extensions for Neural Invariants
• Spatially-Enhanced Temporal Fusion Transformer: Interpretable Multi-Output Prediction for Parametric Dynamical Systems with Time-Varying Inputs
• MissHyper: Restoring Clinical Synchronicity in Missingness-Guided Hypergraph Forecasting
• Learning Population-Level Dynamics through a Latent Fokker--Planck Model and Discrepancy Transport Maps
• SRC-Flow: Compact Semantic Representations Enable Normalizing Flows for Image Generation
• An Introduction to Bayesian and Frequentist Simulation-Based Inference with Machine Learning
• Neural Ordinary Differential Equations for Oscillatory Flows in Aeroelasticity with Application to Transonic Buffet
• Bridging the Gap Between Plausibility and Admissibility: Constraint-Aware Flow Maps for Dynamic Graph Systems
• SlerpFlow: Spherical Trajectory Correction for Rectified Flow Inversion
• A Weak Penalty Neural ODE for Learning Chaotic Dynamics from Noisy Time Series
• Hydrodynamic Backflow for Easing the Fermion Sign in Finite-Temperature Electron Path Integral Simulations
• AG-REPA: Causal Layer Selection for Representation Alignment in Audio Flow Matching
• DINOde: Continuous Vision-Text Alignment for Open-Vocabulary Semantic Segmentation
• A convergence result of a continuous model of deep learning via a Łojasiewicz--Simon inequality
• Native Extrapolation Awareness in Flow-Based Conditional Generation
• Statistical Inference for Generative Model Comparison
• On the Limits of Sampling-Based Reachability: Geometry, Dynamics, and Sample Complexity
• ODeform: Learning Continuous 4D Motion for Shape Deformation with Neural ODEs
• Boltzmann-Expected Molecular Design with Decoupled Annealing Flows
• Nonlinear Bandit
• DeforM: Reasoning-Guided Physics-Aware Video Generation via Spatial-Temporal Masking
• Dynamical Archetype Analysis: Autonomous Computation
• On Transformer Dynamics
• An Adjoint-Sensitivity Framework for Lost-in-the-Middle Phenomena in Causal Residual Transformers
• Neural Kolmogorov Equations: Parallelizable Learning of Stochastic Dynamics under General Noise
• A machine-learned probability distribution in the phase space of turbulent channel flow for synthetic turbulence and flow reconstruction
• Signed Rectified Flow: Negativity-Controlled Generation
• Quantum Port-Hamiltonian Neural Networks: Learning Conservative and Dissipative Dynamics via Measurement-Induced Nonlinearity
• Automatic Construction of Clinical Scoring Systems with LLM Agents
• Knowledge-Informed Kernel State Reconstruction from Heterogeneous Partial Observations
• Equivariant Continuous Normalizing Flows with Offline Sampling for Fermionic Ground State Estimation
• The Geometry of Semantic Space: A Continuous Geometric Framework for the Transformer Architecture
• Holographic Learning from Fermionic Spectra: Application to Strange Metal Phenomenology
• DynImmune-BERT: Dynamic Immune Repertoire Modeling with Neural ODE Driven Continuous Transformers
• Dropout and Random Gradient Masking Are Asymptotically Equivalent in Large ResNets
• TVGL-CFM:Generating and Forecasting Time-Varying Trajectories of Dynamic Networks with Conditional Flow Matching
• Lightweight return-mapping surrogates for multiscale plasticity: a practical guide
• Spatio-Temporal Prediction of Unsteady Airfoil Aerodynamics Using Augmented Graph Neural Ordinary Differential Equations with Exogenous Controls
• Positive-Allocation Companion Predictors for Nonlinear Dynamics and Their Finite-Difference Diagnostics
• Robust Filter Attention: Self-Attention as Precision-Weighted State Estimation
• RTS Smoother-Guided Learning of Physics-Based Neural Differential Models
• Sharp Stability Threshold and Certification for Designing Stable Residual Architectures
• A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems
• Harnessing Machine Learning for Hybrid Constitutive Modelling of Viscoelastic Fluid Flows in Computational Rheology
• A Unified Variational Framework for Optimal Transport with Lagrangian Costs
• Reflex: Real-Time VLA Control through Streaming Inference
• Leveraging Biokinetic Knowledge Priors for Data-Scarce Bioprocess Modeling
• Neural Controlled Differential Equations for EMT-Level Surrogate Modeling of Grid-Forming Inverters
• Information bottleneck for learning the phase space of dynamics from high-dimensional experimental data
• There Will Be a Scientific Theory of Deep Learning
• Position: A Dynamical Systems Perspective is Needed to Advance Time Series Modeling
• Continuous-Time Piecewise-Linear Recurrent Neural Networks
• Beyond networks: Toward adaptive models of biological complexity
• The Mean-Field Dynamics of Transformers
• Multimodal Datasets with Controllable Mutual Information
• Towards a Physics Foundation Model
• Panda: A pretrained forecast model for chaotic dynamics
• True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics
• Adjoint Sampling: Highly Scalable Diffusion Samplers via Adjoint Matching
• NoProp: Training Neural Networks without Full Back-propagation or Full Forward-propagation
• Riemannian Variational Flow Matching for Material and Protein Design
• SDE Matching: Scalable and Simulation-Free Training of Latent Stochastic Differential Equations
• The functional role of oscillatory dynamics in neocortical circuits: A computational perspective
• Sparser, Better, Faster, Stronger: Sparsity Detection for Efficient Automatic Differentiation
• NORi: An ML-Augmented Ocean Boundary Layer Parameterization
• Schrodinger AI: A Unified Spectral-Dynamical Framework for Classification, Reasoning, and Operator-Based Generalization
• Breaking the Memory Wall: Exact Analytical Differentiation via Tiled Operator-Space Evolution
• Quasi-continuum descriptions of rarefaction and dispersive shock waves in Fermi-Pasta-Ulam lattices with Hertzian potentials
• Multimodal Diffeomorphic Registration with Neural ODEs and Structural Descriptors
• Auslander-Yorke Dichotomy and Its Generalizations for Non-Autonomous Dynamical Systems
• Joint Flow Matching for Generator-Consistent Classification
• Generative Modeling by Value-Driven Transport
• Early Detection of Hardware Trojans Using Neural Controlled Differential Equations and Analysis of Power Traces
• Perturbative-NeuSA: A Structured Spectral Framework for Time-Dependent PDEs
• An Adjoint-Based Differentiable Physics Framework for Online Parameter Inversion in Closed-Brayton Gas-Cooled Reactor Digital Twins
• All in One: Generative Modeling as Mean-Field Game Design
• Operator Neural Jump ODEs: L²-optimal prediction in function spaces
• jaxdae: A JAX-native Differentiable Solver for Differential-Algebraic Equations in Coupled Multi-physics
• A Data-Assimilation-Augmented Optimization Framework for Parameter Estimation in Dynamical Systems
• Deep Delta Learning
• Operator learning for models of tear film breakup
• Pro-Pose: Unpaired Full-Body Portrait Synthesis via Canonical UV Maps
• UPMRI: Unsupervised Parallel MRI Reconstruction via Projected Conditional Flow Matching
• HGAN-SDEs: Learning Neural Stochastic Differential Equations with Hermite-Guided Adversarial Training
• Optimality-Informed Neural Networks for Solving Parametric Optimization Problems
• Block-Recurrent Dynamics in Vision Transformers
• The Seismic Wavefield Common Task Framework
• Machine Learning of Temperature-dependent Chemical Kinetics Using Parallel Droplet Microreactors
• 4D Gaussian Splatting as a Learned Dynamical System
• Variational Autoregressive Networks Applied to ϕ⁴ Field Theory Systems
• An Inverse Scattering Inspired Fourier Neural Operator for Time-Dependent PDE Learning
• Robust and scalable simulation-based inference for gravitational wave signals with gaps
• Markovian Promoter Models: A Mechanistic Alternative to Hill Functions in Gene Regulatory Networks
• Is There a Better Source Distribution than Gaussian? Exploring Source Distributions for Image Flow Matching
• A Unified Representation of Neural Networks Architectures
• Generative modeling of conditional probability distributions on the level-sets of collective variables
• DiffeoMorph: Learning to Morph 3D Shapes Using Differentiable Agent-Based Simulations
• FlowDet: Unifying Object Detection and Generative Transport Flows
• CRONOS: Continuous Time Reconstruction for 4D Medical Longitudinal Series
• Higher-Order LaSDI: Reduced Order Modeling with Multiple Time Derivatives
• Time-Frequency Analysis for Neural Networks
• mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAs
• Learning continuous state of charge dependent thermal decomposition kinetics for Li-ion cathodes using Kolmogorov-Arnold Chemical Reaction Neural Networks (KA-CRNNs)
• Softly Constrained Denoisers for Diffusion Models Applied to Partial Differential Equations
• Causal Structure Learning for Dynamical Systems with Theoretical Score Analysis
• Capturing reduced-order quantum many-body dynamics out of equilibrium via neural ordinary differential equations
• Flow matching Operators for Residual-Augmented Probabilistic Learning of Partial Differential Equations
• On Approaches to Building Surrogate ODE Models for Diffusion Bridges
• Fast Online Digital Twinning on FPGA for Mission Critical Applications
• Solving PDEs With Deep Neural Nets under General Boundary Conditions
• Neural CDEs as Correctors for Learned Time Series Models
• Forecasting N-Body Dynamics: A Comparative Study of Neural Ordinary Differential Equations and Universal Differential Equations
• Descent-Net: Learning Descent Directions for Constrained Optimization
• An output scaling layer boosts deep neural networks for multiscale ODE systems
• Bhargava Cube--Inspired Quadratic Regularization for Structured Neural Embeddings
• Neuronal Attention Circuit (NAC) for Representation Learning
• Bidirectional Normalizing Flow: From Data to Noise and Back
• Extrapolation of Periodic Functions Using Binary Encoding of Continuous Numerical Values
• FALCON: Few-step Accurate Likelihoods for Continuous Flows
• CHyLL: Learning Continuous Neural Representations of Hybrid Systems
• A physics-augmented neural network framework for modeling and detecting thermo-visco-plastic behavior
• Neural Ordinary Differential Equations for Simulating Metabolic Pathway Dynamics from Time-Series Multiomics Data
• CFO: Learning Continuous-Time PDE Dynamics via Flow-Matched Neural Operators
• One-Step Diffusion Samplers via Self-Distillation and Deterministic Flow
• Model-Based Diffusion Sampling for Predictive Control in Offline Decision Making
• Repulsor: Accelerating Generative Modeling with a Contrastive Memory Bank
• Augmented Neural Ordinary Differential Equations for Power System Identification
• RRAEDy: Adaptive Latent Linearization of Nonlinear Dynamical Systems
• Interconnection and Damping Assignment Passivity-Based Control using Sparse Neural ODEs
• Optimal and Diffusion Transports in Machine Learning
• Pathway to O(√(d)) Complexity bound under Wasserstein metric of flow-based models
• Control-Oriented System Identification: Classical, Learning, and Physics-Informed Approaches
• Physics-Informed Neural Koopman Machine for Interpretable Longitudinal Personalized Alzheimer's Disease Forecasting
• Hardware Software Optimizations for Fast Model Recovery on Reconfigurable Architectures
• HoloNet: Toward a Unified Einstein-Maxwell-Dilaton Framework of QCD
• FieldSeer I: Physics-Guided World Models for Long-Horizon Electromagnetic Dynamics under Partial Observability
• Virtually Unrolling the Herculaneum Papyri by Diffeomorphic Spiral Fitting
• QuaMo: Quaternion Motions for Vision-based 3D Human Kinematics Capture
• How to Tame Your LLM: Semantic Collapse in Continuous Systems
• PhyVLLM: Physics-Guided Video Language Model with Motion-Appearance Disentanglement
• STeP-Diff: Spatio-Temporal Physics-Informed Diffusion Models for Mobile Fine-Grained Pollution Forecasting
• Exact and Parametric Dynamical System Representation of Nonlinear Functions
• RNNs perform task computations by dynamically warping neural representations
• Joint Distillation for Fast Likelihood Evaluation and Sampling in Flow-based Models
• Model Recovery at the Edge under Resource Constraints for Physical AI
• Much Ado About Noising: Dispelling the Myths of Generative Robotic Control
• Adaptive tumor growth forecasting via neural & universal ODEs
• DiG-Flow: Discrepancy-Guided Flow Matching for Robust VLA Models
• Neural Network Optimal Power Flow via Energy Gradient Flow and Unified Dynamics
• DPAC: Distribution-Preserving Adversarial Control for Diffusion Sampling
• Physics-Constrained Neural Dynamics: A Unified Manifold Framework for Large-Scale Power Flow Computation
• Upper Approximation Bounds for Neural Oscillators
• Optimizing LVLMs with On-Policy Data for Effective Hallucination Mitigation
• Self-sufficient Independent Component Analysis via KL Minimizing Flows
• SmallWorlds: Assessing Dynamics Understanding of World Models in Isolated Environments
• SDE-Attention: Latent Attention in SDE-RNNs for Irregularly Sampled Time Series with Missing Data
• Time Extrapolation with Graph Convolutional Autoencoder and Tensor Train Decomposition
• Normalizing Flows on Quotient Manifolds via Boundary Quotients
• Hard-Constrained Neural Networks with Physics-Embedded Architecture for Residual Dynamics Learning and Invariant Enforcement in Cyber-Physical Systems
• Freeze, Diffuse, Decode: Geometry-Aware Adaptation of Pretrained Transformer Embeddings for Antimicrobial Peptide Design
• Generative Anchored Fields: Controlled Data Generation via Emergent Velocity Fields and Transport Algebra
• Generative models for crystalline materials
• AdS/Deep-Learning made easy II: neural network-based approaches to holography and inverse problems
• Mean-Field Limits for Two-Layer Neural Networks Trained with Consensus-Based Optimization
• Dynamical Implicit Neural Representations
• TSGM: Regular and Irregular Time-series Generation using Score-based Generative Models
• Sparse-to-Field Reconstruction via Stochastic Neural Dynamic Mode Decomposition
• From Features to States: Data-Driven Selection of Measured State Variables via RFE-DMDc
• Parameter-Efficient Neural CDEs via Implicit Function Jacobians
• Alzheimers Disease Progression Prediction Based on Manifold Mapping of Irregularly Sampled Longitudinal Data
• Online Learning-Enhanced High Order Adaptive Safety Control
• EnfoPath: Energy-Informed Analysis of Generative Trajectories in Flow Matching
• VeCoR -- Velocity Contrastive Regularization for Flow Matching
• Breaking the Likelihood-Quality Trade-off in Diffusion Models by Merging Pretrained Experts
• A joint optimization approach to identifying sparse dynamics using least squares kernel collocation
• Foundations of Artificial Intelligence Frameworks: Notion and Limits of AGI
• Modeling Retinal Ganglion Cells with Neural Differential Equations
• ArticFlow: Generative Simulation of Articulated Mechanisms
• Neural Mesh Flow: 3D Manifold Mesh Generation via Diffeomorphic Flows
• ManifoldFormer: Geometric Deep Learning for Neural Dynamics on Riemannian Manifolds
• Saving Foundation Flow-Matching Priors for Inverse Problems
• ODE-ViT: Plug & Play Attention Layer from the Generalization of the ViT as an Ordinary Differential Equation
• Generative Modeling of Clinical Time Series via Latent Stochastic Differential Equations
• Implicit and explicit treatments of model error in numerical simulation
• CODE: A global approach to ODE dynamics learning
• PCARNN-DCBF: Minimal-Intervention Geofence Enforcement for Ground Vehicles
• QML-HCS: A Hypercausal Quantum Machine Learning Framework for Non-Stationary Environments
• Tensor Gauge Flow Models
• STEER: Simple Temporal Regularization For Neural ODEs
• Method of Manufactured Learning for Solver-free Training of Neural Operators
• Hamiltonian Neural Networks
• Neural process model predictive control
• CellStream: Dynamical Optimal Transport Informed Embeddings for Reconstructing Cellular Trajectories from Snapshots Data
• MFI-ResNet: Efficient ResNet Architecture Optimization via MeanFlow Compression and Selective Incubation
• FlowPath: Learning Data-Driven Manifolds with Invertible Flows for Robust Irregularly-sampled Time Series Classification
• Physics-Informed Neural ODEs with Scale-Aware Residuals for Learning Stiff Biophysical Dynamics
• Simulator and Experience Enhanced Diffusion Model for Comprehensive ECG Generation
• Continuum Dropout for Neural Differential Equations
• FlowCast: Advancing Precipitation Nowcasting with Conditional Flow Matching
• When is a System Discoverable from Data? Discovery Requires Chaos
• Generative AI Meets 6G and Beyond: Diffusion Models for Semantic Communications
• Improving Long-Range Interactions in Graph Neural Simulators via Hamiltonian Dynamics
• From Sequential to Recursive: Enhancing Decision-Focused Learning with Bidirectional Feedback
• Accelerating Training Speed of Tiny Recursive Models with Curriculum Guided Adaptive Recursion
• Breaking the Adversarial Robustness-Performance Trade-off in Text Classification via Manifold Purification
• Rectified Noise: A Generative Model Using Positive-incentive Noise
• Kolmogorov-Arnold Chemical Reaction Neural Networks for learning pressure-dependent kinetic rate laws
• Image Restoration via Primal Dual Hybrid Gradient and Flow Generative Model
• Physically-Grounded Goal Imagination: Physics-Informed Variational Autoencoder for Self-Supervised Reinforcement Learning
• Adaptive Initial Residual Connections for GNNs with Theoretical Guarantees
• Latent Refinement via Flow Matching for Training-free Linear Inverse Problem Solving
• Physics-Informed Design of Input Convex Neural Networks for Consistency Optimal Transport Flow Matching
• ITPP: Learning Disentangled Event Dynamics in Marked Temporal Point Processes
• Learning solutions of parameterized stiff ODEs using Gaussian processes
• Towards Frequency-Adaptive Learning for SAR Despeckling
• Zero-Shot Function Encoder-Based Differentiable Predictive Control
• On Flow Matching KL Divergence
• Learning Dynamics from Input-Output Data with Hamiltonian Gaussian Processes
• Synthesizing speech with selected perceptual voice qualities - A case study with creaky voice
• SSTODE: Ocean-Atmosphere Physics-Informed Neural ODEs for Sea Surface Temperature Prediction
• Conditional Neural ODE for Longitudinal Parkinson's Disease Progression Forecasting
• Integrating Score-Based Generative Modeling and Neural ODEs for Accurate Representation of Multiscale Chaotic Dynamics
• Learning Paths for Dynamic Measure Transport: A Control Perspective
• Forecasting Future Anatomies: Longitudinal Brain Mri-to-Mri Prediction
• DoFlow: Causal Generative Flows for Interventional and Counterfactual Time-Series Prediction
• Split-Flows: Measure Transport and Information Loss Across Molecular Resolutions
• RefVTON: person-to-person Try on with Additional Unpaired Visual Reference
• Improving Robustness to Out-of-Distribution States in Imitation Learning via Deep Koopman-Boosted Diffusion Policy
• Sensitivity Analysis for Climate Science with Generative Flow Models
• GeneFlow: Translation of Single-cell Gene Expression to Histopathological Images via Rectified Flow
• Stabilizing Rayleigh-Benard convection with reinforcement learning trained on a reduced-order model
• How Regularization Terms Make Invertible Neural Networks Bayesian Point Estimators
• Curly Flow Matching for Learning Non-gradient Field Dynamics
• Hybrid Physical-Neural Simulator for Fast Cosmological Hydrodynamics
• Reduced order modelling of Hopf bifurcations for the Navier-Stokes equations through invariant manifolds
• Likely Interpolants of Generative Models
• Uncertainty-Aware Diagnostics for Physics-Informed Machine Learning
• Group-Equivariant Diffusion Models for Lattice Field Theory
• Bias-Corrected Data Synthesis for Imbalanced Learning
• Automated Discovery of Conservation Laws via Hybrid Neural ODE-Transformers
• π_RL: Online RL Fine-tuning for Flow-based Vision-Language-Action Models
• Neural Stochastic Flows: Solver-Free Modelling and Inference for SDE Solutions
• Physics-informed continuous normalizing flows to learn the electric field within a time-projection chamber
• Physics-aware Spatiotemporal Modules with Auxiliary Tasks for Meta-Learning
• Neural Controlled Differential Equations for Online Prediction Tasks
• MetaKoopman: Bayesian Meta-Learning of Koopman Operators for Modeling Structured Dynamics under Distribution Shifts
• Deep learning of parameterized equations with applications to uncertainty quantification
• ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEs
• Autoregressive Boltzmann Generators
• Distance Marching for Generative Modeling
• RGMem: Renormalization Group-inspired Memory Evolution for Language Agents
• Sparse Transformer Architectures via Regularized Wasserstein Proximal Operator with L₁ Prior
• Hybrid Liquid Neural Network-Random Finite Set Filtering for Robust Maneuvering Object Tracking
• Bridging Simulators with Conditional Optimal Transport
• Sample-efficient and Scalable Exploration in Continuous-Time RL
• Unsupervised Detection of Post-Stroke Brain Abnormalities
• Graph-Guided Network for Irregularly Sampled Multivariate Time Series
• Identifiable learning of dissipative dynamics
• Simulation-free Structure Learning for Stochastic Dynamics
• Protein Folding with Neural Ordinary Differential Equations
• Common Task Framework For a Critical Evaluation of Scientific Machine Learning Algorithms
• Nested AutoRegressive Models
• Stable neural networks and connections to continuous dynamical systems
• TRASE-NODEs: Trajectory Sensitivity-aware Neural Ordinary Differential Equations for Efficient Dynamic Modeling
• Dynamic Graph Neural Network for Data-Driven Physiologically Based Pharmacokinetic Modeling
• A Differentiable Solver Approach to Operator Inference
• EditInfinity: Image Editing with Binary-Quantized Generative Models
• Target-aware Image Editing via Cycle-consistent Constraints
• Speculative Sampling for Parametric Temporal Point Processes
• Towards Interpretable Deep Learning and Analysis of Dynamical Systems via the Discrete Empirical Interpolation Method
• Generative Unfolding of Jets and Their Substructure
• A Survey on Cache Methods in Diffusion Models: Toward Efficient Multi-Modal Generation
• LyTimeT: Towards Robust and Interpretable State-Variable Discovery
• Non-intrusive structural-preserving sequential data assimilation
• Layer-Parallel Training of Deep Residual Neural Networks
• On Biologically Plausible Learning in Continuous Time
• Iterative Refinement of Flow Policies in Probability Space for Online Reinforcement Learning
• Discovering How Ice Crystals Grow Using Neural ODE's and Symbolic Regression
• Deep Neural Network extraction of Unpolarized Transverse Momentum Distributions
• Temporally Detailed Hypergraph Neural ODEs for Disease Progression Modeling
• Fine-tuning Flow Matching Generative Models with Intermediate Feedback
• Deep Neural ODE Operator Networks for PDEs
• Still Competitive: Revisiting Recurrent Models for Irregular Time Series Prediction
• Revealing Low-Dimensional Structure in 2D Richtmyer-Meshkov Instabilities via Parametric Reduced-Order Modeling
• EARS-UDE: Evaluating Auditory Response in Sensory Overload with Universal Differential Equations
• AlignFlow: Improving Flow-based Generative Models with Semi-Discrete Optimal Transport
• LeapFactual: Reliable Visual Counterfactual Explanation Using Conditional Flow Matching
• A Structured Neural ODE Approach for Real Time Evaluation of AC Losses in 3D Superconducting Tapes
• Functional and parametric identifiability for universal differential equations applied to chemical reaction networks
• Data-driven Soliton Manifold Approximations for Dark and Bright Waves: Some Prototypical 1d Case Examples
• Convergence, design and training of continuous-time dropout as a random batch method
• Emergence of Internal State-Modulated Swarming in Multi-Agent Patch Foraging System
• Unconditional Human Motion and Shape Generation via Balanced Score-Based Diffusion
• E-MoFlow: Learning Egomotion and Optical Flow from Event Data via Implicit Regularization
• Efficient Inference for Coupled Hidden Markov Models in Continuous Time and Discrete Space
• Y-shaped Generative Flows
• Dimension-Free Minimax Rates for Learning Pairwise Interactions in Attention-Style Models
• Analyzing Data Quality and Decay in Mega-Constellations: A Physics-Informed Machine Learning Approach
• Rough Path Signatures: Learning Neural RDEs for Portfolio Optimization
• Moser Flow: Divergence-based Generative Modeling on Manifolds
• Fine-Tuning Flow Matching via Maximum Likelihood Estimation of Reconstructions
• Adaptive Heterogeneous Mixtures of Normalising Flows for Robust Variational Inference
• Mathematical Modeling and Convergence Analysis of Deep Neural Networks with Dense Layer Connectivities in Deep Learning
• Reinforcement Fine-Tuning of Flow-Matching Policies for Vision-Language-Action Models
• Artificial intelligence as a surrogate brain: Bridging neural dynamical models and data
• Neural non-canonical Hamiltonian dynamics for long-time simulations
• cuHPX: GPU-Accelerated Differentiable Spherical Harmonic Transforms on HEALPix Grids
• A Physical Theory of Backpropagation: Exact Gradients from the Least-Action Principle
• Temporal Lifting as Latent-Space Regularization for Continuous-Time Flow Models in AI Systems
• HeunNet: Extending ResNet using Heun's Methods
• Physics-Informed High-order Graph Dynamics Identification Learning for Predicting Complex Networks Long-term Dynamics
• FLOWING: Implicit Neural Flows for Structure-Preserving Morphing
• Score-Based Density Estimation from Pairwise Comparisons
• Conditional Flow Matching for Bayesian Posterior Inference
• MODE: Learning compositional representations of complex systems with Mixtures Of Dynamical Experts
• Operator Learning for Power Systems Simulation
• Counterfactual Identifiability via Dynamic Optimal Transport
• Kelp: A Streaming Safeguard for Large Models via Latent Dynamics-Guided Risk Detection
• FC2T2: The Fast Continuous Convolutional Taylor Transform with Applications in Vision and Graphics
• DM1: MeanFlow with Dispersive Regularization for 1-Step Robotic Manipulation
• Deep Neural Networks Inspired by Differential Equations
• Rectified-CFG++ for Flow Based Models
• Wavefunction Flows: Efficient Quantum Simulation of Continuous Flow Models
• Bridging the Physics-Data Gap with FNO-Guided Conditional Flow Matching: Designing Inductive Bias through Hierarchical Physical Constraints
• Transferable Generative Models Bridge Femtosecond to Nanosecond Time-Step Molecular Dynamics
• Accuracy, Memory Efficiency and Generalization: A Comparative Study on Liquid Neural Networks and Recurrent Neural Networks
• SympNets: Intrinsic structure-preserving symplectic networks for identifying Hamiltonian systems
• Diffusion Models and the Manifold Hypothesis: Log-Domain Smoothing is Geometry Adaptive
• Application of deep neural networks for computing the renormalization group flow of the two-dimensional phi⁴ field theory
• Lagrangian neural ODEs: Measuring the existence of a Lagrangian with Helmholtz metrics
• Thermodynamic Performance Limits for Score-Based Diffusion Models
• Carré du champ flow matching: better quality-generalisation tradeoff in generative models
• Stable Robot Motions on Manifolds: Learning Lyapunov-Constrained Neural Manifold ODEs
• PointNSP: Autoregressive 3D Point Cloud Generation with Next-Scale Level-of-Detail Prediction
• Fundamental Limits of Crystalline Equivariant Graph Neural Networks: A Circuit Complexity Perspective
• Physics-Informed Machine Learning in Biomedical Science and Engineering
• Control of dynamical systems with neural networks
• Collective dynamics on higher-order networks
• DISCO-DJ II: a differentiable particle-mesh code for cosmology
• Stiff Circuit System Modeling via Transformer
• AD-NODE: Adaptive Dynamics Learning with Neural ODEs for Mobile Robots Control
• Efficient Manifold-Constrained Neural ODE for High-Dimensional Datasets
• Nyström-Accelerated Primal LS-SVMs: Breaking the O(an³) Complexity Bottleneck for Scalable ODEs Learning
• A Mathematical Explanation of Transformers for Large Language Models and GPTs
• Modeling Time Series Dynamics with Fourier Ordinary Differential Equations
• Wave-PDE Nets: Trainable Wave-Equation Layers as an Alternative to Attention
• On the Convergence and Size Transferability of Continuous-depth Graph Neural Networks
• Proximal Diffusion Neural Sampler
• Achieving Universal Approximation and Universal Interpolation via Nonlinearity of Control Families
• Latent Mixture of Symmetries for Sample-Efficient Dynamic Learning
• ContextFlow: Context-Aware Flow Matching For Trajectory Inference From Spatial Omics Data
• Learning Explicit Single-Cell Dynamics Using ODE Representations
• Deep Generative Continual Learning using Functional LoRA: FunLoRA
• Longitudinal Flow Matching for Trajectory Modeling
• Dynamical system reconstruction from partial observations using stochastic dynamics
• Riemannian Consistency Model
• Physics-Informed Neural Controlled Differential Equations for Scalable Long Horizon Multi-Agent Motion Forecasting
• Time-series forecasting with deep learning: a survey
• F-scheduler: illuminating the free-lunch design space for fast sampling of diffusion models
• Differentiable Autoencoding Neural Operator for Interpretable and Integrable Latent Space Modeling
• WAN3DNS: Weak Adversarial Networks for Solving 3D Incompressible Navier-Stokes Equations
• Neural Network State-Space Estimators
• Act to See, See to Act: Diffusion-Driven Perception-Action Interplay for Adaptive Policies
• Reweighted Flow Matching via Unbalanced OT for Label-free Long-tailed Generation
• Biologically inspired neural network layer with homeostatic regulation and adaptive repair mechanisms
• Flow Matching with Semidiscrete Couplings
• Scalable Boltzmann Generators for equilibrium sampling of large-scale materials
• Let Physics Guide Your Protein Flows: Topology-aware Unfolding and Generation
• A multiscale analysis of mean-field transformers in the moderate interaction regime
• MarS-FM: Generative Modeling of Molecular Dynamics via Markov State Models
• In-Context Learning of Temporal Point Processes with Foundation Inference Models
• VNODE: A Piecewise Continuous Volterra Neural Network
• Learning to Solve Optimization Problems Constrained with Partial Differential Equations
• A study of Universal ODE approaches to predicting soil organic carbon
• Anatomy-DT: A Cross-Diffusion Digital Twin for Anatomical Evolution
• SoC-DT: Standard-of-Care Aligned Digital Twins for Patient-Specific Tumor Dynamics
• Semantic Editing with Coupled Stochastic Differential Equations
• Diffusion Schr "odinger Bridge with Applications to Score-Based\n Generative Modeling
• Advantage Weighted Matching: Aligning RL with Pretraining in Diffusion Models
• Position-Blind Ptychography: Viability of image reconstruction via data-driven variational inference
• StrucADT: Generating Structure-controlled 3D Point Clouds with Adjacency Diffusion Transformer
• A Neural ODE Approach to Aircraft Flight Dynamics Modelling
• MeanFlowSE: One-Step Generative Speech Enhancement via MeanFlow
• On the design space between molecular mechanics and machine learning force fields
• Dynamics of Learning: Generative Schedules from Latent ODEs
• IIET: Efficient Numerical Transformer via Implicit Iterative Euler Method
• Error Analysis of Discrete Flow with Generator Matching
• Physically Plausible Multi-System Trajectory Generation and Symmetry Discovery
• Model reduction of parametric ordinary differential equations via autoencoders: structure-preserving latent dynamics and convergence analysis
• A Causality-Aware Spatiotemporal Model for Multi-Region and Multi-Pollutant Air Quality Forecasting
• MeanSE: Efficient Generative Speech Enhancement with Mean Flows
• MORPH: PDE Foundation Models with Arbitrary Data Modality
• NewtonGen: Physics-Consistent and Controllable Text-to-Video Generation via Neural Newtonian Dynamics
• PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generation
• Time-adaptive HénonNets for separable Hamiltonian systems
• Staying on the Manifold: Geometry-Aware Noise Injection
• Formal Safety Verification and Refinement for Generative Motion Planners via Certified Local Stabilization
• Metriplectic Conditional Flow Matching for Dissipative Dynamics
• Looped Transformers with Source-Centered State Evolution
• Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) network
• Persistent Gaussian Perturbations Prevent Oversmoothing in Recurrent Graph Neural Networks
• Enhancing Irregular Time Series Forecasting with Continuous-Time Modeling Framework
• ODEWorld: A Continuous Predictive Architecture via Physical-Time Flow
• Physics-constrained neural ordinary differential equation models to discover and predict microbial community dynamics
• Neural Ordinary Differential Equations for Data-Driven Reduced Order\n Modeling of Environmental Hydrodynamics
• Efficient Explicit Taylor ODE Integrators with Symbolic-Numeric Computing
• Linear Power System Modeling and Analysis Across Wide Operating Ranges: A Hierarchical Neural State-Space Equation Approach
• Prompt-Guided Dual Latent Steering for Inversion Problems
• MTM: A Multi-Scale Token Mixing Transformer for Irregular Multivariate Time Series Classification
• Preconditioned Deformation Grids
• Breaking the Discretization Barrier of Continuous Physics Simulation Learning
• Emergent Topology of Optimal Networks for Synchrony
• Dynamic Gene Regulatory Network Inference with Interpretable, Biophysically-Motivated Neural ODEs
• Preference Trajectory Modeling via Flow Matching for Sequential Recommendation
• ViTCAE: ViT-based Class-conditioned Autoencoder
• Time-adaptive SympNets for separable Hamiltonian systems
• Modeling Irregular Astronomical Time Series with Neural Stochastic Delay Differential Equations
• TANDEM: Temporal Attention-guided Neural Differential Equations for Missingness in Time Series Classification
• Quantum neural ordinary and partial differential equations
• MeanFlowSE: one-step generative speech enhancement via conditional mean flow
• FlowCast-ODE: Continuous Hourly Weather Forecasting with Dynamic Flow Matching and ODE Solver
• Evidential Physics-Informed Neural Networks for Scientific Discovery
• CDFlow: Generative Gradient Flows for Configuration Space Distance Fields via Neural ODEs
• Floating-Body Hydrodynamic Neural Networks
• Spatiotemporal graph neural process for reconstruction, extrapolation, and classification of cardiac trajectories
• Reversible Deep Equilibrium Models
• Universal differential equations for systems biology: Current state and open problems
• Path Integral Sampler: a stochastic control approach for sampling
• Deep Convolutional Networks as shallow Gaussian Processes
• Learning non-Markovian Dynamical Systems with Signature-based Encoders
• C3DE: Causal-Aware Collaborative Neural Controlled Differential Equation for Long-Term Urban Crowd Flow Prediction
• Zero to Autonomy in Real-Time: Online Adaptation of Dynamics in Unstructured Environments
• Gradients, parallelism, and variance of quantum estimates
• Competitive Inner-Imaging Squeeze and Excitation for Residual Network
• Two ways to knowledge?
• Flow Straight and Fast in Hilbert Space: Functional Rectified Flow
• KAN-SR: A Kolmogorov-Arnold Network Guided Symbolic Regression Framework
• SciML Agents: Write the Solver, Not the Solution
• The Hidden Width of Deep ResNets: Tight Error Bounds and Phase Diagram
• Uncertainty Propagation Networks for Neural Ordinary Differential Equations
• Sig-DEG for Distillation: Making Diffusion Models Faster and Lighter
• Improving Video Diffusion Transformer Training by Multi-Feature Fusion and Alignment from Self-Supervised Vision Encoders
• SAFT: Shape and Appearance of Fabrics from Template via Differentiable Physical Simulations from Monocular Video
• LatentVoiceGrad: Nonparallel Voice Conversion with Latent Diffusion/Flow-Matching Models
• OCTANE -- Optimal Control for Tensor-based Autoencoder Network Emergence: Explicit Case
• Unstructured to structured: geometric multigrid on complex geometries via domain remapping
• AI LLM Proof of Self-Consciousness and User-Specific Attractors
• BULL-ODE: Bullwhip Learning with Neural ODEs and Universal Differential Equations under Stochastic Demand
• Physics-Informed Neural Networks in Clean Combustion: A Pathway to Sustainable Aerospace Propulsion
• An Agentic AI Workflow to Simplify Parameter Estimation of Complex Differential Equation Systems
• Learning Generalized Hamiltonian Dynamics with Stability from Noisy Trajectory Data
• Quantum Filtering and Stabilization of Dissipative Quantum Systems via Augmented Neural Ordinary Differential Equations
• Data-driven discovery of dynamical models in biology
• A High-order Backpropagation Algorithm for Neural Stochastic Differential Equation Model
• Any-Step Density Ratio Estimation via Interval-Annealed Secant Alignment
• Neuro-Spectral Architectures for Causal Physics-Informed Networks
• STL-based Optimization of Biomolecular Neural Networks for Regression and Control
• Echo State Networks as State-Space Models: A Systems Perspective
• Error analysis for learning the time-stepping operator of evolutionary PDEs
• Balancing Signal and Variance: Adaptive Offline RL Post-Training for VLA Flow Models
• An invertible generative model for forward and inverse problems
• Neural Architecture Search via Bregman Iterations
• Energy-Weighted Flow Matching: Unlocking Continuous Normalizing Flows for Efficient and Scalable Boltzmann Sampling
• Distribution estimation via Flow Matching with Lipschitz guarantees
• Beyond Ensembles: Simulating All-Atom Protein Dynamics in a Learned Latent Space
• Exploring accuracy and uncertainty quantification in physics-informed neural networks for inferring microbial community dynamics
• Augmented KRnet for density estimation and approximation
• Discrete Noise Inversion for Next-scale Autoregressive Text-based Image Editing
• Preconditioned Regularized Wasserstein Proximal Sampling
• Learning Longitudinal Stress Dynamics from Irregular Self-Reports via Time Embeddings
• HyperPINN: Learning parameterized differential equations with physics-informed hypernetworks
• Continuous Graph Neural Networks
• Cross-Domain Few-Shot Segmentation via Ordinary Differential Equations over Time Intervals
• Disentangling Slow and Fast Temporal Dynamics in Degradation Inference with Hierarchical Differential Models
• Are We Really Learning the Score Function? Reinterpreting Diffusion Models Through Wasserstein Gradient Flow Matching
• Universal Representation of Generalized Convex Functions and their Gradients
• Estimating Parameter Fields in Multi-Physics PDEs from Scarce Measurements
• FNODE: Flow-Matching for data-driven simulation of constrained multibody systems
• Generative Latent Space Dynamics of Electron Density
• OptMark: Robust Multi-bit Diffusion Watermarking via Inference Time Optimization
• Learning to Discretize: Solving 1D Scalar Conservation Laws via Deep Reinforcement Learning
• Studying Effective String Theory using deep generative models
• Self-Composing Neural Operators with Depth and Accuracy Scaling via Adaptive Train-and-Unroll Approach
• Flowing Straighter with Conditional Flow Matching for Accurate Speech Enhancement
• Neural Field Turing Machine: A Differentiable Spatial Computer
• Objective Value Change and Shape-Based Accelerated Optimization for the Neural Network Approximation
• FlowMalTrans: Unsupervised Binary Code Translation for Malware Detection Using Flow-Adapter Architecture
• Data-Augmented Few-Shot Neural Emulator for Computer-Model System Identification
• Kolmogorov-Arnold Representation for Symplectic Learning: Advancing Hamiltonian Neural Networks
• STDiff: A State Transition Diffusion Framework for Time Series Imputation in Industrial Systems
• Energy-Based Flow Matching for Generating 3D Molecular Structure
• Generalization Bound for a General Class of Neural Ordinary Differential Equations
• From Prediction to Simulation: AlphaFold 3 as a Differentiable Framework for Structural Biology
• Amortized Sampling with Transferable Normalizing Flows
• Provable Mixed-Noise Learning with Flow-Matching
• Incorporating Pre-trained Diffusion Models in Solving the Schrödinger Bridge Problem
• Sesame: Opening the door to protein pockets
• Lorentz-Equivariance without Limitations
• On Scalable and Efficient Computation of Large Scale Optimal Transport
• Counterfactual Probabilistic Diffusion with Expert Models
• A Tutorial on Deep Latent Variable Models of Natural Language
• Distribution Matching via Generalized Consistency Models
• Cosmology-informed Neural Networks to infer dark energy equation-of-state
• Robust Convolution Neural ODEs via Contractivity-promoting regularization
• Learning State-Space Models of Dynamic Systems from Arbitrary Data using Joint Embedding Predictive Architectures
• Observable Optimization for Precision Theory: Machine Learning Energy Correlators
• Starobinsky in Stereo: SKA-CMB Synergy in SBI
• Forecasting Black Sigatoka Infection Risks with Latent Neural ODEs
• GeoMAE: Masking Representation Learning for Spatio-Temporal Graph Forecasting with Missing Values
• Implicit Hypergraph Neural Networks: A Stable Framework for Higher-Order Relational Learning with Provable Guarantees
• Real-time forecasting of chaotic dynamics from sparse data and autoencoders
• Identifying efficient routes to laminarization: an optimization approach
• Forecasting Continuous Non-Conservative Dynamical Systems in SO(3)
• Momentum Point-Perplexity Mechanics in Large Language Models
• When and how can inexact generative models still sample from the data manifold?
• Intrinsic training dynamics of deep neural networks
• Neural Bridge Processes
• FlowSE: Flow Matching-based Speech Enhancement
• Structure-Preserving Digital Twins via Conditional Neural Whitney Forms
• Towards High-Order Mean Flow Generative Models: Feasibility, Expressivity, and Provably Efficient Criteria
• A Stage-Aware Mixture of Experts Framework for Neurodegenerative Disease Progression Modelling
• Artificial intelligence in mitotic checkpoint modeling: transforming our understanding of cellular division through machine learning and predictive biology
• Learning geometries beyond asymptotic AdS
• Machine Learning-Based Nonlinear Nudging for Chaotic Dynamical Systems
• How and Why: Taming Flow Matching for Unsupervised Anomaly Detection and Localization
• Estimating Musical Surprisal from Audio in Autoregressive Diffusion Model Noise Spaces
• TANGO: Graph Neural Dynamics via Learned Energy and Tangential Flows
• Optimality Principles and Neural Ordinary Differential Equations-based Process Modeling for Distributed Control
• Multi-Marginal Stochastic Flow Matching for High-Dimensional Snapshot Data at Irregular Time Points
• Lie Transform--based Neural Networks for Dynamics Simulation and\n Learning
• Variety Is the Spice of Life: Detecting Misinformation with Dynamic Environmental Representations
• SCFlow: Implicitly Learning Style and Content Disentanglement with Flow Models
• How Diffusion Prior Landscapes Shape the Posterior in Blind Deconvolution
• REFLECT: Rectified Flows for Efficient Brain Anomaly Correction Transport
• Physics-Embedded Neural ODEs for Sim2Real Edge Digital Twins of Hybrid Power Electronics Systems
• DeepKoopFormer: A Koopman Enhanced Transformer Based Architecture for Time Series Forecasting
• Diffusion models for inverse problems
• Epi²-Net: Advancing Epidemic Dynamics Forecasting with Physics-Inspired Neural Networks
• Revitalizing Canonical Pre-Alignment for Irregular Multivariate Time Series Forecasting
• Instance-Dependent Continuous-Time Reinforcement Learning via Maximum Likelihood Estimation
• CTBench: Cryptocurrency Time Series Generation Benchmark
• Neural Predictive Control to Coordinate Discrete- and Continuous-Time Models for Time-Series Analysis with Control-Theoretical Improvements
• VFP: Variational Flow-Matching Policy for Multi-Modal Robot Manipulation
• The Vanishing Gradient Problem for Stiff Neural Differential Equations
• Unified Generation-Refinement Planning: Bridging Guided Flow Matching and Sampling-Based MPC for Social Navigation
• Flow Matching for Probabilistic Learning of Dynamical Systems from Missing or Noisy Data
• WeightFlow: Learning Stochastic Dynamics via Evolving Weight of Neural Network
• Unlocking New Paths for Science with Extreme-Mass-Ratio Inspirals: Machine Learning-Enhanced MCMC for Accurate Parameter Inversion
• TrajSurv: Learning Continuous Latent Trajectories from Electronic Health Records for Trustworthy Survival Prediction
• Stability-Constrained AC Optimal Power Flow--A Gaussian Process-Based Approach
• Image-Guided Shape-from-Template Using Mesh Inextensibility Constraints
• Weighted Conditional Flow Matching
• Learning Simulatable Models of Cloth with Spatially-varying Constitutive Properties
• Numerical PDE solvers outperform neural PDE solvers
• Explicit and Effectively Symmetric Runge-Kutta Methods
• Why Flow Matching is Particle Swarm Optimization?
• Neural differential equation [wikipedia]
• Flow-based generative model [wikipedia]
• Adjoint state method [wikipedia]
• David Duvenaud [wikipedia]

Discussions

• Neural Ordinary Differential Equations [hn, 240 points, 60 comments]
• Neural Ordinary Differential Equations [lobsters, 10 points, 1 comments]
• Neural Ordinary Differential Equations [hn, 3 points, 0 comments]
• oh ya, that makes more sense. i really want to get back to this stuff. this paper would be so much fun to try implementing in julia (no idea what has happened in this space in the last 4-5 years) ar [bsky, 1 points, 1 comments]
• arxiv.org/abs/1806.07366 [bsky, 0 points, 0 comments]

Related

• Dissecting Neural ODEs
• Do Residual Neural Networks discretize Neural Ordinary Differential Equations?
• Differential Equations for Continuous-Time Deep Learning
• Generalization bounds for neural ordinary differential equations and deep residual networks
• Continuous Convolutional Neural Networks: Coupled Neural PDE and ODE