2019/02/01 by Dinghan Shen, Shen, Dinghan, Asli Celikyilmaz +12 · 3 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling #cs.CL #cs.LG
paper · pdf · doi:10.48550/arxiv.1902.00154
To appear at ACL 2019
openalex publication_date 2019/02/01 · openalex created_date 2019/02/21 · arxiv created 2019/06/19 · arxiv updated 2019/06/21 · openalex updated_date 2026/07/28
Variational autoencoders (VAEs) have received much attention recently as an end-to-end architecture for text generation with latent variables. In this paper, we investigate several multi-level structures to learn a VAE model to generate long, and coherent text. In particular, we use a hierarchy of stochastic layers between the encoder and decoder networks to generate more informative latent codes. We also investigate a multi-level decoder structure to learn a coherent long-term structure by generating intermediate sentence representations as high-level plan vectors. Empirical results demonstrate that a multi-level VAE model produces more coherent and less repetitive long text compared to the standard VAE models and can further mitigate the posterior-collapse issue.