Masked Autoregressive Flow for Density Estimation

2017/05/19 by George Papamakarios, Papamakarios, George, Theo Pavlakou +3 · 204 citations
Computer Science · Mathematics · Physics and Astronomy · #Adversarial Robustness in Machine Learning #Applied mathematics #Autoregressive integrated moving average #Autoregressive model #Computer science #Econometrics #Estimator #FOS: Computer and information sciences #Flow (mathematics) #Generalization #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Mathematics #Model Reduction and Neural Networks #Nonlinear autoregressive exogenous model #SETAR #STAR model #Statistics #Time series #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1705.07057

published in arXiv (Cornell University) 30, 2338-2347 (Cornell University) · section 4.3 is corrected since the previous version

openalex publication_date 2017/05/19 · arxiv created 2018/06/14 · arxiv updated 2018/06/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

Autoregressive models are among the best performing neural density estimators. We describe an approach for increasing the flexibility of an autoregressive model, based on modelling the random numbers that the model uses internally when generating data. By constructing a stack of autoregressive models, each modelling the random numbers of the next model in the stack, we obtain a type of normalizing flow suitable for density estimation, which we call Masked Autoregressive Flow. This type of flow is closely related to Inverse Autoregressive Flow and is a generalization of Real NVP. Masked Autoregressive Flow achieves state-of-the-art performance in a range of general-purpose density estimation tasks.

Citations

• Auto-Encoding Variational Bayes
• PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications
• Fast ε-free Inference of Simulation Models with Bayesian Conditional Density Estimation
• Learning Multiple Layers of Features from Tiny Images

Cited by

• Inherited or produced? Inferring protein production kinetics when protein counts are shaped by a cell's division history
• Latent Sculpting for Zero-Shot Generalization: A Manifold Learning Approach to Out-of-Distribution Anomaly Detection
• Minimum Distance Summaries for Robust Neural Posterior Estimation
• Normalizing Trajectory Models
• Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Models
• Improving Matrix Exponential for Generative AI Flows: A Taylor-Based Approach Beyond Paterson--Stockmeyer
• Inverse Autoregressive Flows for Zero Degree Calorimeter fast simulation
• Robust and scalable simulation-based inference for gravitational wave signals with gaps
• Implicit Likelihood Inference of the Neutrino Mass Hierarchy from Cosmological Data
• Transport Reversible Jump Markov Chain Monte Carlo with proposals generated by Variational Inference with Normalizing Flows
• Bidirectional Normalizing Flow: From Data to Noise and Back
• chatter: a Python library for applying information theory and AI/ML models to animal communication
• ClearPotential: Revealing Local Dark Matter in Three Dimensions
• Forecasting Dark Matter Subhalo Constraints from Stellar Streams using Implicit Likelihood Inference
• State Diversity Matters in Offline Behavior Distillation
• Amortized Inference of Multi-Modal Posteriors using Likelihood-Weighted Normalizing Flows
• SimFlow: Simplified and End-to-End Training of Latent Normalizing Flows
• TARFVAE: Efficient One-Step Generative Time Series Forecasting via TARFLOW based VAE
• STARFlow-V: End-to-End Video Generative Modeling with Normalizing Flows
• CroTad: A Contrastive Reinforcement Learning Framework for Online Trajectory Anomaly Detection
• Generative Pricing of Basket Options via Signature-Conditioned Mixture Density Networks
• DiScoFormer: Plug-In Density and Score Estimation with Transformers
• Dark Energy Survey Year 3 results: Simulation-based wCDM inference from weak lensing and galaxy clustering maps with deep learning: Analysis design
• Accelerated Sequential Posterior Inference via Reuse for Gravitational-Wave Analyses
• Addressing prior dependence in hierarchical Bayesian modeling for PTA data analysis I: Methodology and implementation
• Diffusion Models at the Drug Discovery Frontier: A Review on Generating Small Molecules versus Therapeutic Peptides
• IrisML: Neural Posterior Estimation for the Spectral Energy Distribution fitting
• CDFlow: Building Invertible Layers with Circulant and Diagonal Matrices
• FARMER: Flow AutoRegressive Transformer over Pixels
• Variational Polya Tree
• HollowFlow: Efficient Sample Likelihood Evaluation using Hollow Message Passing
• Neural Mutual Information Estimation with Vector Copulas
• Beyond single tracers: CNN-based inference of galaxy mass profiles from combined gas and stellar kinematics
• Matricial Free Energy as a Gaussianizing Regularizer: Enhancing Autoencoders for Gaussian Code Generation
• SBi3PCF: Simulation-based inference with the integrated 3PCF
• Adaptive Heterogeneous Mixtures of Normalising Flows for Robust Variational Inference
• Efficient Autoregressive Inference for Transformer Probabilistic Models
• Simulation-based inference for neutrino interaction model parameter tuning
• HybridFlow: Quantification of Aleatoric and Epistemic Uncertainty with a Single Hybrid Model
• CINDES: Classification induced neural density estimator and simulator
• One-shot Conditional Sampling: MMD meets Nearest Neighbors
• GDR-learners: Orthogonal Learning of Generative Models for Potential Outcomes
• Scale-Wise VAR is Secretly Discrete Diffusion
• GenUQ: Predictive Uncertainty Estimates via Generative Hyper-Networks
• Random Projection Flows for Efficient Manifold Density Estimation
• Segmental copy number amplifications are more stable than aneuploidies in the absence of selection
• Flow-Induced Diagonal Gaussian Processes
• Simulation-based Inference of Massive Black Hole Binaries using Sequential Neural Likelihood
• Modeling nonstationary spatial processes with normalizing flows
• TabResFlow: A Normalizing Spline Flow Model for Probabilistic Univariate Tabular Regression
• Trends in the Population of Binary Black Holes Following the Fourth Gravitational-Wave Transient Catalog: a Data-Driven Analysis
• Conditional Density Estimation with Bayesian Normalising Flows
• Neural Posterior Unfolding
• Misspecification-robust amortised simulation-based inference using variational methods
• ML-PWS: Estimating the Mutual Information Between Experimental Time Series Using Neural Networks
• Simulation based inference of the ionization history from the 2D 21 cm power spectrum
• Augmented KRnet for density estimation and approximation
• Data-driven mean-field within whole-brain models
• Simulation-Based Inference for Direction Reconstruction of Ultra-High-Energy Cosmic Rays with Radio Arrays
• FlowMalTrans: Unsupervised Binary Code Translation for Malware Detection Using Flow-Adapter Architecture
• Fractal Flow: Hierarchical and Interpretable Normalizing Flow via Topic Modeling and Recursive Strategy
• Bayesian Insights into Exchange and Restriction in Gray Matter Diffusion MRI
• On Surjectivity of Neural Networks: Can you elicit any behavior from your model?
• Amortized Sampling with Transferable Normalizing Flows
• Mining gold from implicit models to improve likelihood-free inference
• A Tutorial on Deep Latent Variable Models of Natural Language
• Observable Optimization for Precision Theory: Machine Learning Energy Correlators
• Sensitivity Analysis to Unobserved Confounding with Copula-based Normalizing Flows
• Towards Robust Active Feature Acquisition
• In situ estimation of the acoustic surface impedance using simulation-based inference
• CaPulse: Detecting Anomalies by Tuning in to the Causal Rhythms of Time Series
• Trustworthy scientific inference with generative models
• Quantitative and Predictive Folding Models from Limited Single-Molecule Data Using Simulation-Based Inference
• How to embed any likelihood into SBI: Application to Planck + Stage IV galaxy surveys and Dynamical Dark Energy
• Comparing Normalizing Flows with Kernel Density Estimation in Estimating Risk of Automated Driving Systems
• The Constitutional Controller: Doubt-Calibrated Steering of Compliant Agents
• Joint Inference of Trajectory and Obstacle in Mean-Field Games via Bilevel Optimization
• Flow Stochastic Segmentation Networks
• Featurized Density Ratio Estimation
• The Role of Snowmelt and Subsurface Heterogeneity in Headwater Hydrology of a Mountainous Catchment in Colorado: A Model‐Data Integration Approach
• Flow Matching Meets Biology and Life Science: A Survey
• Manifold-preserved GANs
• Neural Human Pose Prior
• Recent Advances in Simulation-based Inference for Gravitational Wave Data Analysis
• VAE-KRnet and its applications to variational Bayes
• Estimating Marginal Likelihoods in Likelihood-Free Inference via Neural Density Estimation
• Normalizing Flow to Augmented Posterior: Conditional Density Estimation with Interpretable Dimension Reduction for High Dimensional Data
• Latent Thermodynamic Flows: Unified Representation Learning and Generative Modeling of Temperature-Dependent Behaviors from Limited Data
• A Real-Time Digital Twin for Type 1 Diabetes using Simulation-Based Inference
• Generative flow-based warm start of the variational quantum eigensolver
• Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flows
• Simulation-based population inference of LISA's Galactic binaries: Bypassing the global fit
• Distilling Normalizing Flows
• Direct Fisher Score Estimation for Likelihood Maximization
• Data-Driven High-Dimensional Statistical Inference with Generative Models
• STARFlow: Scaling Latent Normalizing Flows for High-resolution Image Synthesis
• Primordial Black Holes and the First Stars
• Multilevel neural simulation-based inference
• Variational Transdimensional Inference
• Learning Balanced Field Summaries of the Large-Scale Structure with the Neural Field Scattering Transform
• Mitigating mode collapse in normalizing flows by annealing with an adaptive schedule: Application to parameter estimation
• Accelerating Inference of Discrete Autoregressive Normalizing Flows by Selective Jacobi Decoding
• BIMA: Bijective Maximum Likelihood Learning Approach to Hallucination Prediction and Mitigation in Large Vision-Language Models
• Model-Informed Flows for Bayesian Inference
• Generative modelling of multivariate geometric extremes using normalising flows
• Normalizing Flows are Capable Models for Continuous Control
• DiCoFlex: Model-agnostic diverse counterfactuals with flexible control
• A thermodynamic approach to Approximate Bayesian Computation with multiple summary statistics
• JAPAN: Joint Adaptive Prediction Areas with Normalising-Flows
• Causal Posterior Estimation
• Wavelet Flow For Extragalactic Foreground Simulations
• Serial-OE: Anomalous sound detection based on serial method with outlier exposure capable of using small amounts of anomalous data for training
• Improving Anomalous Sound Detection through Pseudo-anomalous Set Selection and Pseudo-label Utilization under Unlabeled Conditions
• Fixing a Broken ELBO
• Multivariate Latent Recalibration for Conditional Normalizing Flows
• On-Demand Scenario Generation for Testing Automated Driving Systems
• Hybrid Bernstein Normalizing Flows for Flexible Multivariate Density Regression with Interpretable Marginals
• Accelerate TarFlow Sampling with GS-Jacobi Iteration
• On Measuring Intrinsic Causal Attributions in Deep Neural Networks
• Continuous Temporal Learning of Probability Distributions via Neural ODEs with Applications in Continuous Glucose Monitoring Data
• Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy
• Adaptive Kernel Density Estimation with Pre-training
• Offline Multi-agent Reinforcement Learning via Sequential Score Decomposition
• Distilling Two-Timed Flow Models by Separately Matching Initial and Terminal Velocities
• BayesFlow 2: Multi-Backend Amortized Bayesian Inference in Python
• Modeling the Gaia Color-Magnitude Diagram with Bayesian Neural Flows to Constrain Distance Estimates
• Using Variational Inference to Improve the Efficiency of MCMC Algorithms
• Latent Reasoning with Normalizing Flows
• Neural Density Estimation and Likelihood-free Inference
• Profiling systematic uncertainties in Simulation-Based Inference with Factorizable Normalizing Flows
• Trajectory-Oriented Optimization Via Adaptive Thompson Sampling And Grid Refinement: A Tutorial With The ADAPTIVE_TS Package
• Truncated Neural Likelihood Estimation for Simulation-Based Inference in State-Space Models
• Effortless, Simulation-Efficient Bayesian Inference using Tabular Foundation Models
• Rare and Different: Anomaly Scores from a combination of likelihood and out-of-distribution models to detect new physics at the LHC
• A novel stellar spectrum denoising method based on deep Bayesian modeling
• Improving Sequential Latent Variable Models with Autoregressive Flows
• Image Editing with Diffusion Models: A Survey
• Reducing Calls to the Simulator in Simulation Based Inference (SBI)
• Potential Matching Optimal Transport: Continuous Normalizing Flows for Exact p-Wasserstein Dynamics
• Normalizing Flow Regression for Bayesian Inference with Offline Likelihood Evaluations
• A real-time anomaly detection method for robots based on a flexible and sparse latent space
• Transforming Simulation to Data Without Pairing

Related

• Block Neural Autoregressive Flow
• Neural Autoregressive Flows
• A Triangular Network For Density Estimation
• Probabilistic Classification by Density Estimation Using Gaussian Mixture Model and Masked Autoregressive Flow
• Transformer Neural Autoregressive Flows