2023/04/04 by Christopher Zach, Zach, Christopher
Computer Science · Mathematics · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications #Statistical Methods and Inference #Time Series Analysis and Forecasting
paper · pdf · doi:10.48550/arxiv.2304.02473
openalex publication_date 2023/04/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
By using the underlying theory of proper scoring rules, we design a family of noise-contrastive estimation (NCE) methods that are tractable for latent variable models. Both terms in the underlying NCE loss, the one using data samples and the one using noise samples, can be lower-bounded as in variational Bayes, therefore we call this family of losses fully variational noise-contrastive estimation. Variational autoencoders are a particular example in this family and therefore can be also understood as separating real data from synthetic samples using an appropriate classification loss. We further discuss other instances in this family of fully variational NCE objectives and indicate differences in their empirical behavior.