Towards Foundation Inference Models that Learn ODEs In-Context

2025/10/14 by Mauel, Maximilian, Patrick Seifner, Hinz, Manuel +5 · 2 citations
Computer Science · Engineering · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reservoir Engineering and Simulation Methods #Semantic Web and Ontologies

paper · pdf · doi:10.48550/arxiv.2510.12650

openalex publication_date 2025/10/14 · openalex created_date 2025/10/17 · openalex updated_date 2026/07/28

Abstract

Ordinary differential equations (ODEs) describe dynamical systems evolving deterministically in continuous time. Accurate data-driven modeling of systems as ODEs, a central problem across the natural sciences, remains challenging, especially if the data is sparse or noisy. We introduce FIM-ODE (Foundation Inference Model for ODEs), a pretrained neural model designed to estimate ODEs zero-shot (i.e., in context) from sparse and noisy observations. Trained on synthetic data, the model utilizes a flexible neural operator for robust ODE inference, even from corrupted data. We empirically verify that FIM-ODE provides accurate estimates, on par with a neural state-of-the-art method, and qualitatively compare the structure of their estimated vector fields.

Citations

• In-Context Learning of Stochastic Differential Equations with Foundation Inference Models
• Foundation Inference Models for Markov Jump Processes
• Zero-shot Imputation with Foundation Inference Models for Dynamical Systems
• ODEFormer: Symbolic Regression of Dynamical Systems with Transformers
• Neural Dynamic Focused Topic Model
• The future is different: Large pre-trained language models fail in prediction tasks
• Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators
• Learning interpretable continuous-time models of latent stochastic dynamical systems
• Learning unknown ODE models with Gaussian processes
• Deterministic Nonperiodic Flow

Cited by

• On Foundation Models for Temporal Point Processes to Accelerate Scientific Discovery
• Towards Fast Coarse-graining and Equation Discovery with Foundation Inference Models
• In-Context Learning of Temporal Point Processes with Foundation Inference Models

Related

• Learning the Simplest Neural ODE
• Zero-shot Imputation with Foundation Inference Models for Dynamical Systems
• Neural ODEs with Irregular and Noisy Data
• Beyond Predictions in Neural ODEs: Identification and Interventions
• CODE: A global approach to ODE dynamics learning