On the Convergence Rate of Gaussianization with Random Rotations

2023/06/23 by Felix Draxler, Draxler, Felix, Lars Kühmichel +9 · 2 citations
Computer Science · #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Music and Audio Processing #Speech and Audio Processing

paper · pdf · doi:10.48550/arxiv.2306.13520

openalex publication_date 2023/06/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Gaussianization is a simple generative model that can be trained without backpropagation. It has shown compelling performance on low dimensional data. As the dimension increases, however, it has been observed that the convergence speed slows down. We show analytically that the number of required layers scales linearly with the dimension for Gaussian input. We argue that this is because the model is unable to capture dependencies between dimensions. Empirically, we find the same linear increase in cost for arbitrary input p(x), but observe favorable scaling for some distributions. We explore potential speed-ups and formulate challenges for further research.

Cited by

• Matricial Free Energy as a Gaussianizing Regularizer: Enhancing Autoencoders for Gaussian Code Generation
• Rotated Mean-Field Variational Inference and Iterative Gaussianization

Related

• Rethinking Gauss-Newton for learning over-parameterized models
• Randomly Projected Additive Gaussian Processes for Regression
• Rates of Convergence for Sparse Variational Gaussian Process Regression
• When does Gaussian equivalence fail and how to fix it: Non-universal behavior of random features with quadratic scaling
• Euclidean algorithms are Gaussian