Diagnosing and Enhancing VAE Models

2019/03/14 by Bin Dai, David Wipf, Dai, Bin +1 · 15 citations
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Music and Audio Processing

paper · pdf · doi:10.48550/arxiv.1903.05789

openalex publication_date 2019/03/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Although variational autoencoders (VAEs) represent a widely influential deep generative model, many aspects of the underlying energy function remain poorly understood. In particular, it is commonly believed that Gaussian encoder/decoder assumptions reduce the effectiveness of VAEs in generating realistic samples. In this regard, we rigorously analyze the VAE objective, differentiating situations where this belief is and is not actually true. We then leverage the corresponding insights to develop a simple VAE enhancement that requires no additional hyperparameters or sensitive tuning. Quantitatively, this proposal produces crisp samples and stable FID scores that are actually competitive with a variety of GAN models, all while retaining desirable attributes of the original VAE architecture. A shorter version of this work will appear in the ICLR 2019 conference proceedings (Dai and Wipf, 2019). The code for our model is available at https://github.com/daib13/ TwoStageVAE.

Citations

• Importance Weighted Autoencoders
• Auto-Encoding Variational Bayes
• Gradient-based learning applied to document recognition
• Independent component analysis: algorithms and applications
• Representation Learning: A Review and New Perspectives
• Tutorial on Variational Autoencoders
• Variational Lossy Autoencoder
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step
• On Convergence and Stability of GANs
• Demystifying MMD GANs
• Conditional Image Generation with PixelCNN Decoders
• Learning Multiple Layers of Features from Tiny Images

Cited by

• A Multi-stage Constrained Optimization Framework for Data-driven Problems
• Disentangled representations via score-based variational autoencoders
• VAEVQ: Enhancing Discrete Visual Tokenization through Variational Modeling
• An efficient probabilistic hardware architecture for diffusion-like models
• Learning Latent Energy-Based Models via Interacting Particle Langevin Dynamics
• Learning Energy-based Variational Latent Prior for VAEs
• FishAI 2.0: Marine Fish Image Classification with Multi-modal Few-shot Learning
• Random Projection Flows for Efficient Manifold Density Estimation
• Momentum Contrastive Autoencoder: Using Contrastive Learning for Latent Space Distribution Matching in WAE
• Generalizing Variational Autoencoders with Hierarchical Empirical Bayes
• CbLDM: A Diffusion Model for recovering nanostructure from atomic pair distribution function
• Quantum-Boosted High-Fidelity Deep Learning
• Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse
• Flow Matching for Probabilistic Learning of Dynamical Systems from Missing or Noisy Data
• ControlVAE: Tuning, Analytical Properties, and Performance Analysis

Related

• How to train your VAE
• A Tutorial on VAEs: From Bayes' Rule to Lossless Compression
• Re-parameterizing VAEs for stability
• Simple and Effective VAE Training with Calibrated Decoders
• Energy-Calibrated VAE with Test Time Free Lunch