2021/11/22 by Laurence Illing Midgley, Midgley, Laurence Illing, Vincent Stimper +5
Computer Science · Mathematics · Physics and Astronomy · #Adversarial Robustness in Machine Learning #Algorithm #Bootstrapping (finance) #Computer science #Divergence (linguistics) #Econometrics #Flow (mathematics) #Gaussian Processes and Bayesian Inference #Mathematical optimization #Mathematics #Model Reduction and Neural Networks #Parameterized complexity #Sampling (signal processing) #Variance (accounting) #cs.AI #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.2111.11510
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2021/11/22 · arxiv created 2022/03/14 · arxiv updated 2022/03/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Normalizing flows are flexible, parameterized distributions that can be used to approximate expectations from intractable distributions via importance sampling. However, current flow-based approaches are limited on challenging targets where they either suffer from mode seeking behaviour or high variance in the training loss, or rely on samples from the target distribution, which may not be available. To address these challenges, we combine flows with annealed importance sampling (AIS), while using the α-divergence as our objective, in a novel training procedure, FAB (Flow AIS Bootstrap). Thereby, the flow and AIS improve each other in a bootstrapping manner. We demonstrate that FAB can be used to produce accurate approximations to complex target distributions, including Boltzmann distributions, in problems where previous flow-based methods fail.