Disfluency Detection using a Noisy Channel Model and a Deep Neural Language Model

2018/08/27 by Paria Jamshid Lou, Mark Johnson, Lou, Paria Jamshid +1 · 1 citation
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection #Speech Recognition and Synthesis #Speech and Audio Processing #cs.CL

paper · pdf · doi:10.48550/arxiv.1808.09091

openalex publication_date 2018/08/27 · arxiv created 2018/08/28 · arxiv updated 2018/08/29 · openalex created_date 2022/08/03 · openalex updated_date 2026/07/28

Abstract

This paper presents a model for disfluency detection in spontaneous speech transcripts called LSTM Noisy Channel Model. The model uses a Noisy Channel Model (NCM) to generate n-best candidate disfluency analyses and a Long Short-Term Memory (LSTM) language model to score the underlying fluent sentences of each analysis. The LSTM language model scores, along with other features, are used in a MaxEnt reranker to identify the most plausible analysis. We show that using an LSTM language model in the reranking process of noisy channel disfluency model improves the state-of-the-art in disfluency detection.

Cited by

• Z-Scores: A Metric for Linguistically Assessing Disfluency Removal

Related

• Disfluency Detection using Auto-Correlational Neural Networks
• Disfluency Detection using a Bidirectional LSTM
• Auxiliary Sequence Labeling Tasks for Disfluency Detection
• Improving Disfluency Detection by Self-Training a Self-Attentive Model
• Disfluency Detection with Unlabeled Data and Small BERT Models