Neural Net Models for Open-Domain Discourse Coherence

2016/06/05 by Jiwei Li, Dan Jurafsky, Li, Jiwei +1 · 3 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1606.01545

openalex publication_date 2016/06/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Discourse coherence is strongly associated with text quality, making it important to natural language generation and understanding. Yet existing models of coherence focus on measuring individual aspects of coherence (lexical overlap, rhetorical structure, entity centering) in narrow domains. In this paper, we describe domain-independent neural models of discourse coherence that are capable of measuring multiple aspects of coherence in existing sentences and can maintain coherence while generating new sentences. We study both discriminative models that learn to distinguish coherent from incoherent discourse, and generative models that produce coherent text, including a novel neural latent-variable Markovian generative model that captures the latent discourse dependencies between sentences in a text. Our work achieves state-of-the-art performance on multiple coherence evaluations, and marks an initial step in generating coherent texts given discourse contexts.

Citations

• ADADELTA: An Adaptive Learning Rate Method
• A Neural Conversational Model
• Learning Stochastic Recurrent Networks
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method
• Modeling Local Coherence: An Entity-Based Approach
• Contextual LSTM (CLSTM) models for Large scale NLP tasks
• Tutorial on Variational Autoencoders
• Adversarial Evaluation of Dialogue Models
• Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Models

Cited by

• Decoding Dynamic Visual Experience from Calcium Imaging via Cell-Pattern-Aware Pretraining
• Comparison of Unsupervised Metrics for Evaluating Judicial Decision Extraction
• DENSE: Longitudinal Progress Note Generation with Temporal Modeling of Heterogeneous Clinical Notes Across Hospital Visits

Related

• Discourse Coherence in the Wild: A Dataset, Evaluation and Methods
• Is Incoherence Surprising? Targeted Evaluation of Coherence Prediction\n from Language Models
• Neural RST-based Evaluation of Discourse Coherence
• A Unified Neural Coherence Model
• A Cross-Domain Transferable Neural Coherence Model