A Deep Generative Framework for Paraphrase Generation

2017/09/15 by Ankush Gupta, Arvind Agarwal, Gupta, Ankush +5 · 134 citations
Computer Science · #Artificial intelligence #Benchmark (surveying) #Computer science #Conversation #Deep learning #Generative grammar #Generative model #Linguistics #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Natural language processing #Paraphrase #Sentence #Sequence (biology) #Topic Modeling #cs.CL

paper · pdf · doi:10.48550/arxiv.1709.05074

published in arXiv (Cornell University) 32(1), 5149-5156 (Cornell University)

arxiv created 2017/09/15 · openalex publication_date 2017/09/15 · arxiv updated 2017/09/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Paraphrase generation is an important problem in NLP, especially in question answering, information retrieval, information extraction, conversation systems, to name a few. In this paper, we address the problem of generating paraphrases automatically. Our proposed method is based on a combination of deep generative models (VAE) with sequence-to-sequence models (LSTM) to generate paraphrases, given an input sentence. Traditional VAEs when combined with recurrent neural networks can generate free text but they are not suitable for paraphrase generation for a given sentence. We address this problem by conditioning the both, encoder and decoder sides of VAE, on the original sentence, so that it can generate the given sentence's paraphrases. Unlike most existing models, our model is simple, modular and can generate multiple paraphrases, for a given sentence. Quantitative evaluation of the proposed method on a benchmark paraphrase dataset demonstrates its efficacy, and its performance improvement over the state-of-the-art methods by a significant margin, whereas qualitative human evaluation indicate that the generated paraphrases are well-formed, grammatically correct, and are relevant to the input sentence. Furthermore, we evaluate our method on a newly released question paraphrase dataset, and establish a new baseline for future research.

Cited by

• How to Ask Better Questions? A Large-Scale Multi-Domain Dataset for Rewriting Ill-Formed Questions
• Polly Want a Cracker: Analyzing Performance of Parroting on Paraphrase Generation Datasets
• Quda: Natural Language Queries for Visual Data Analytics
• Crisscrossed Captions: Extended Intramodal and Intermodal Semantic Similarity Judgments for MS-COCO
• Delexicalized Paraphrase Generation
• From Solving a Problem Boldly to Cutting the Gordian Knot: Idiomatic Text Generation
• Contrastive Representation Learning for Exemplar-Guided Paraphrase Generation
• PAWS: Paraphrase Adversaries from Word Scrambling
• Generating Sentences from Disentangled Syntactic and Semantic Spaces
• Exploring Neural Models for Parsing Natural Language into First-Order Logic
• Variational Inference for Learning Representations of Natural Language Edits
• Exemplar-Controllable Paraphrasing and Translation using Bitext
• Unsupervised Paraphrase Generation using Pre-trained Language Models
• Non-autoregressive Transformer by Position Learning
• Learning Semantic Sentence Embeddings using Sequential Pair-wise Discriminator
• Paraphrase Generation as Unsupervised Machine Translation
• Unsupervised Text Generation by Learning from Search
• SMCLM: Semantically Meaningful Causal Language Modeling for Autoregressive Paraphrase Generation
• CGMH: Constrained Sentence Generation by Metropolis-Hastings Sampling
• Unsupervised Contextual Paraphrase Generation using Lexical Control and Reinforcement Learning
• Essentia: Mining Domain-Specific Paraphrases with Word-Alignment Graphs
• Unsupervised Paraphrasing with Pretrained Language Models
• Unsupervised Paraphrasing by Simulated Annealing
• Generating Syntactically Controlled Paraphrases without Using Annotated Parallel Pairs
• Paraphrase Generation with Deep Reinforcement Learning
• Exploring Fluent Query Reformulations with Text-to-Text Transformers and Reinforcement Learning
• Generating Diverse Translation by Manipulating Multi-Head Attention
• Improving Non-autoregressive Generation with Mixup Training
• Syntax-Infused Variational Autoencoder for Text Generation
• Ouroboros: On Accelerating Training of Transformer-Based Language Models

Related

• Paraphrase Generation with Deep Reinforcement Learning
• ReDecode Framework for Iterative Improvement in Paraphrase Generation
• Neural Paraphrase Generation with Stacked Residual LSTM Networks
• Unsupervised Paraphrase Generation using Pre-trained Language Models
• Vector-Quantized Prompt Learning for Paraphrase Generation