Posterior Collapse of a Linear Latent Variable Model

2022/05/09 by Zihao Wang, Liu Ziyin, Wang, Zihao +1 · 1 citation
Computer Science · #Bayesian Methods and Mixture Models #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.2205.04009

openalex publication_date 2022/05/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This work identifies the existence and cause of a type of posterior collapse that frequently occurs in the Bayesian deep learning practice. For a general linear latent variable model that includes linear variational autoencoders as a special case, we precisely identify the nature of posterior collapse to be the competition between the likelihood and the regularization of the mean due to the prior. Our result suggests that posterior collapse may be related to neural collapse and dimensional collapse and could be a subclass of a general problem of learning for deeper architectures.

Cited by

• Posterior Collapse as a Phase Transition in Variational Autoencoders

Related

• The Prevalence of Neural Collapse in Neural Multivariate Regression
• Limitations of Neural Collapse for Understanding Generalization in Deep Learning
• Neural collapse with unconstrained features
• Unifying Low Dimensional Spectra in Deep Learning
• Exact Phase Transitions in Deep Learning