Assessing the Use of Prosody in Constituency Parsing of Imperfect Transcripts

2021/06/14 by Trang Tran, Tran, Trang, Mari Ostendorf +1
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Speech Recognition and Synthesis #Speech and dialogue systems #cs.CL

paper · pdf · doi:10.48550/arxiv.2106.07794

Interspeech 2021

arxiv created 2021/06/14 · openalex publication_date 2021/06/14 · arxiv updated 2021/06/16 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28

Abstract

This work explores constituency parsing on automatically recognized transcripts of conversational speech. The neural parser is based on a sentence encoder that leverages word vectors contextualized with prosodic features, jointly learning prosodic feature extraction with parsing. We assess the utility of the prosody in parsing on imperfect transcripts, i.e. transcripts with automatic speech recognition (ASR) errors, by applying the parser in an N-best reranking framework. In experiments on Switchboard, we obtain 13-15% of the oracle N-best gain relative to parsing the 1-best ASR output, with insignificant impact on word recognition error rate. Prosody provides a significant part of the gain, and analyses suggest that it leads to more grammatical utterances via recovering function words.

Related

• Parsing Speech: A Neural Approach to Integrating Lexical and Acoustic-Prosodic Information
• Neural Constituency Parsing of Speech Transcripts
• On the Role of Style in Parsing Speech with Neural Models
• Prosodic segmentation for parsing spoken dialogue
• Prosodic features improve sentence segmentation and parsing