All-in-one simulation-based inference

2024/04/15 by Manuel Gloeckler, Gloeckler, Manuel, Michael Deistler +7 · 2 voices · 12 citations
Decision Sciences · #Simulation Techniques and Applications #cs.AI #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2404.09636

openalex publication_date 2024/04/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Amortized Bayesian inference trains neural networks to solve stochastic inference problems using model simulations, thereby making it possible to rapidly perform Bayesian inference for any newly observed data. However, current simulation-based amortized inference methods are simulation-hungry and inflexible: They require the specification of a fixed parametric prior, simulator, and inference tasks ahead of time. Here, we present a new amortized inference method -- the Simformer -- which overcomes these limitations. By training a probabilistic diffusion model with transformer architectures, the Simformer outperforms current state-of-the-art amortized inference approaches on benchmark tasks and is substantially more flexible: It can be applied to models with function-valued parameters, it can handle inference scenarios with missing or unstructured data, and it can sample arbitrary conditionals of the joint distribution of parameters and data, including both posterior and likelihood. We showcase the performance and flexibility of the Simformer on simulators from ecology, epidemiology, and neuroscience, and demonstrate that it opens up new possibilities and application domains for amortized Bayesian inference on simulation-based models.

Cited by

• Minimum Distance Summaries for Robust Neural Posterior Estimation
• Flexible Gravitational-Wave Parameter Estimation with Transformers
• MIST: Mutual Information Estimation Via Supervised Training
• Fast and Robust Simulation-Based Inference With Optimization Monte Carlo
• Scalable Simulation-Based Model Inference with Test-Time Complexity Control
• PriorGuide: Test-Time Prior Adaptation for Simulation-Based Inference
• Efficient Autoregressive Inference for Transformer Probabilistic Models
• Score-Based Density Estimation from Pairwise Comparisons
• Multi-modal Foundation Model for Cosmological Simulation Data
• Unified Spatiotemporal Physics-Informed Learning (USPIL): A Framework for Modeling Complex Predator-Prey Dynamics
• Misspecification-robust amortised simulation-based inference using variational methods
• Provable Mixed-Noise Learning with Flow-Matching

Discussions

• I always find the figures from the @mackelab.bsky.social really clear and pretty! For example in arxiv.org/abs/2404.09636. [bsky, 12 points, 1 comments]
• 3) And, last but certainly not least, to improve, scale and generalise simulation-based inference #SBI for discovering scientific models form data --- e.g. taking the Simformer as a starting point a [bsky, 4 points, 1 comments]

Related

• Neural Methods for Amortized Inference
• Compositional simulation-based inference for time series
• Neural Architectures for Amortized Bayesian Inference: Statistical Foundations and Empirical Assessments
• Generalized Bayesian Inference for Scientific Simulators via Amortized Cost Estimation
• BayesFlow: Amortized Bayesian Workflows With Neural Networks