Variational Transdimensional Inference

2025/06/05 by Laurence Davies, Daniel MacKinlay, Davies, Laurence +5
Computer Science · #Computation (stat.CO) #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Generative Adversarial Networks and Image Synthesis #Machine Learning (stat.ML) #Methodology (stat.ME) #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2506.04749

openalex publication_date 2025/06/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

The expressiveness of flow-based models combined with stochastic variational inference (SVI) has expanded the application of optimization-based Bayesian inference to highly complex problems. However, despite the importance of multi-model Bayesian inference for problems defined on a transdimensional joint model and parameter space, such as Bayesian structure learning and model selection, flow-based SVI has been limited to problems defined on a fixed-dimensional parameter space. We introduce CoSMIC, normalizing flows (COntextually-Specified Masking for Identity-mapped Components), an extension to neural autoregressive conditional normalizing flow architectures that enables use of a single flow-based variational density for inference over a transdimensional (multi-model) conditional target distribution. We propose a combined stochastic variational transdimensional inference (VTI) approach to training CoSMIC, flows using ideas from Bayesian optimization and Monte Carlo gradient estimation. Numerical experiments show the performance of VTI on challenging problems that scale to high-cardinality model spaces.

Citations

• Variational Search Distributions
• Neural Methods for Amortized Inference
• Mixed Variational Flows for Discrete Variables
• Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Network
• Comparison of Affine and Rational Quadratic Spline Coupling and Autoregressive Flows through Robust Statistical Tests
• Transport Reversible Jump Proposals
• DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterization
• DiBS: Differentiable Bayesian Structure Learning
• Simulations evaluating resampling methods for causal discovery: ensemble\n performance and calibration
• Learning Likelihoods with Conditional Normalizing Flows
• Flow Models for Arbitrary Conditional Likelihoods
• Monte Carlo Gradient Estimation in Machine Learning
• Neural Spline Flows
• Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows
• Proximal Policy Optimization Algorithms
• Masked Autoregressive Flow for Density Estimation
• Categorical Reparameterization with Gumbel-Softmax
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Improving Variational Inference with Inverse Autoregressive Flow
• Variational Inference: A Review for Statisticians
• Variational Inference with Normalizing Flows
• Transdimensional Transformation based Markov Chain Monte Carlo
• Black Box Variational Inference
• Automated Variational Inference in Probabilistic Programming
• Stochastic Variational Inference
• Variational Bayesian Inference with Stochastic Search
• Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems
• Reversible jump Markov chain Monte Carlo and multi-model samplers
• Taking the Human Out of the Loop: A Review of Bayesian Optimization

Related

• Automatic variational inference with cascading flows
• Stochastic Normalizing Flows
• Improving Variational Inference with Inverse Autoregressive Flow
• Tensorizing flows: a tool for variational inference
• Flow Matching for Scalable Simulation-Based Inference