2019/12/11 by Harshvardhan Sikka, Sikka, Harshvardhan, Weishun Zhong +6 · 1 citation
Computer Science · Decision Sciences · Physics and Astronomy · #Advanced Data Storage Technologies #Big Data Technologies and Applications #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Generative Adversarial Networks and Image Synthesis #Internet Traffic Analysis and Secure E-voting #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Model Reduction and Neural Networks
paper · pdf · doi:10.48550/arxiv.1912.05127
openalex publication_date 2019/12/11 · openalex created_date 2024/04/11 · openalex updated_date 2026/07/28
In many data analysis tasks, it is beneficial to learn representations where each dimension is statistically independent and thus disentangled from the others. If data generating factors are also statistically independent, disentangled representations can be formed by Bayesian inference of latent variables. We examine a generalization of the Variational Autoencoder (VAE), β-VAE, for learning such representations using variational inference. β-VAE enforces conditional independence of its bottleneck neurons controlled by its hyperparameter β. This condition is in general not compatible with the statistical independence of latents. By providing analytical and numerical arguments, we show that this incompatibility leads to a non-monotonic inference performance in β-VAE with a finite optimal β.