2020/10/13 by Ghazi Felhi, Felhi, Ghazi, Joseph Le Roux +4
Social Sciences · Computer Science · #Computational and Text Analysis Methods #Generative Adversarial Networks and Image Synthesis #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2010.06549
Even though Variational Autoencoders (VAEs) are widely used for\nsemi-supervised learning, the reason why they work remains unclear. In fact,\nthe addition of the unsupervised objective is most often vaguely described as a\nregularization. The strength of this regularization is controlled by\ndown-weighting the objective on the unlabeled part of the training set. Through\nan analysis of the objective of semi-supervised VAEs, we observe that they use\nthe posterior of the learned generative model to guide the inference model in\nlearning the partially observed latent variable. We show that given this\nobservation, it is possible to gain finer control on the effect of the\nunsupervised objective on the training procedure. Using importance weighting,\nwe derive two novel objectives that prioritize either one of the partially\nobserved latent variable, or the unobserved latent variable. Experiments on the\nIMDB english sentiment analysis dataset and on the AG News topic classification\ndataset show the improvements brought by our prioritization mechanism and\nexhibit a behavior that is inline with our description of the inner working of\nSemi-Supervised VAEs.\n