vix.ing · top · new · best · stats

Neural Discrete Representation Learning

2017/11/02 by Aaron van den Oord, Aäron van den Oord, Oord, Aaron van den +4 · 2 voices · 1,207 citations
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Music and Audio Processing #Speech Recognition and Synthesis #Speech and Audio Processing #cs.LG

paper · pdf · doi:10.48550/arxiv.1711.00937

openalex publication_date 2017/11/02 · arxiv published 2017/11/02 · arxiv created 2018/05/30 · arxiv updated 2018/05/31 · openalex created_date 2019/07/30 · openalex updated_date 2026/07/28

Abstract

Learning useful representations without supervision remains a key challenge in machine learning. In this paper, we propose a simple yet powerful generative model that learns such discrete representations. Our model, the Vector Quantised-Variational AutoEncoder (VQ-VAE), differs from VAEs in two key ways: the encoder network outputs discrete, rather than continuous, codes; and the prior is learnt rather than static. In order to learn a discrete latent representation, we incorporate ideas from vector quantisation (VQ). Using the VQ method allows the model to circumvent issues of "posterior collapse" -- where the latents are ignored when they are paired with a powerful autoregressive decoder -- typically observed in the VAE framework. Pairing these representations with an autoregressive prior, the model can generate high quality images, videos, and speech as well as doing high quality speaker conversion and unsupervised learning of phonemes, providing further evidence of the utility of the learnt representations.

Cited by

Discussions

Related