2024/02/09 by Felix Draxler, Draxler, Felix, Stefan Wahl +5 · 5 citations
Decision Sciences · Economics, Econometrics and Finance · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Simulation Techniques and Applications #Stochastic processes and financial applications
paper · pdf · doi:10.48550/arxiv.2402.06578
openalex publication_date 2024/02/09 · openalex created_date 2024/02/13 · openalex updated_date 2026/07/28
We present a novel theoretical framework for understanding the expressive power of normalizing flows. Despite their prevalence in scientific applications, a comprehensive understanding of flows remains elusive due to their restricted architectures. Existing theorems fall short as they require the use of arbitrarily ill-conditioned neural networks, limiting practical applicability. We propose a distributional universality theorem for well-conditioned coupling-based normalizing flows such as RealNVP. In addition, we show that volume-preserving normalizing flows are not universal, what distribution they learn instead, and how to fix their expressivity. Our results support the general wisdom that affine and related couplings are expressive and in general outperform volume-preserving flows, bridging a gap between empirical results and theoretical understanding.