Disentangling Latent Space for VAE by Label Relevant/Irrelevant Dimensions

2018/12/22 by Zhilin Zheng, Zheng, Zhilin, Li Sun +1 · 1 citation
Computer Science · #Advanced Image and Video Retrieval Techniques #Computer Vision and Pattern Recognition (cs.CV) #Digital Media Forensic Detection #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis

paper · pdf · doi:10.48550/arxiv.1812.09502

openalex publication_date 2018/12/22 · openalex created_date 2019/06/27 · openalex updated_date 2026/07/28

Abstract

VAE requires the standard Gaussian distribution as a prior in the latent space. Since all codes tend to follow the same prior, it often suffers the so-called "posterior collapse". To avoid this, this paper introduces the class specific distribution for the latent code. But different from CVAE, we present a method for disentangling the latent space into the label relevant and irrelevant dimensions, \bmz_s and \bmz_u, for a single input. We apply two separated encoders to map the input into \bmz_s and \bmz_u respectively, and then give the concatenated code to the decoder to reconstruct the input. The label irrelevant code \bmz_u represent the common characteristics of all inputs, hence they are constrained by the standard Gaussian, and their encoder is trained in amortized variational inference way, like VAE. While \bmz_s is assumed to follow the Gaussian mixture distribution in which each component corresponds to a particular class. The parameters for the Gaussian components in \bmz_s encoder are optimized by the label supervision in a global stochastic way. In theory, we show that our method is actually equivalent to adding a KL divergence term on the joint distribution of \bmz_s and the class label c, and it can directly increase the mutual information between \bmz_s and the label c. Our model can also be extended to GAN by adding a discriminator in the pixel domain so that it produces high quality and diverse images.

Citations

• Learning Face Representation from Scratch
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Auto-Encoding Variational Bayes
• ImageNet Large Scale Visual Recognition Challenge
• Adversarial Feature Learning
• f-GAN: Training Generative Neural Samplers using Variational Divergence\n Minimization
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Semi-Amortized Variational Autoencoders
• cGANs with Projection Discriminator
• FD-GAN: Pose-guided Feature Distilling GAN for Robust Person Re-identification
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• Learning Multiple Layers of Features from Tiny Images

Cited by

• Self-supervised Latent Space Optimization with Nebula Variational Coding

Related

• PartitionVAE -- a human-interpretable VAE
• Disentangling Variational Autoencoders
• Improving Conditional VAE with approximation using Normalizing Flows
• Variational Diffusion Auto-encoder: Latent Space Extraction from Pre-trained Diffusion Models
• Disentanglement as Identifiable Pushforward Factorisation