2020/01/07 by Yao Fu, Fu, Yao, Yansong Feng +3 · 6 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2001.01941
openalex publication_date 2020/01/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Paraphrase generation is a longstanding important problem in natural language processing. In addition, recent progress in deep generative models has shown promising results on discrete latent variables for text generation. Inspired by variational autoencoders with discrete latent structures, in this work, we propose a latent bag of words (BOW) model for paraphrase generation. We ground the semantics of a discrete latent variable by the BOW from the target sentences. We use this latent variable to build a fully differentiable content planning and surface realization model. Specifically, we use source words to predict their neighbors and model the target BOW with a mixture of softmax. We use Gumbel top-k reparameterization to perform differentiable subset sampling from the predicted BOW distribution. We retrieve the sampled word embeddings and use them to augment the decoder and guide its generation search space. Our latent BOW model not only enhances the decoder, but also exhibits clear interpretability. We show the model interpretability with regard to (i) unsupervised learning of word neighbors (ii) the step-by-step generation procedure. Extensive experiments demonstrate the transparent and effective generation process of this model.\footnoteOur code can be found at \urlhttps://github.com/FranxYao/dgmlatentbow