2019/04/17 by Paria Jamshid Lou, Yufei Wang, Lou, Paria Jamshid +3 · 2 citations
Computer Science · #Artificial intelligence #Computation and Language (cs.CL) #Computer science #FOS: Computer and information sciences #Natural Language Processing Techniques #Natural language processing #Parsing #Punctuation #Speech and dialogue systems #Speech recognition #Task (project management) #Topic Modeling #cs.CL
paper · pdf · doi:10.48550/arxiv.1904.08535
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2019/04/17 · arxiv created 2020/04/10 · arxiv updated 2020/04/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06
This paper studies the performance of a neural self-attentive parser on transcribed speech. Speech presents parsing challenges that do not appear in written text, such as the lack of punctuation and the presence of speech disfluencies (including filled pauses, repetitions, corrections, etc.). Disfluencies are especially problematic for conventional syntactic parsers, which typically fail to find any EDITED disfluency nodes at all. This motivated the development of special disfluency detection systems, and special mechanisms added to parsers specifically to handle disfluencies. However, we show here that neural parsers can find EDITED disfluency nodes, and the best neural parsers find them with an accuracy surpassing that of specialized disfluency detection systems, thus making these specialized mechanisms unnecessary. This paper also investigates a modified loss function that puts more weight on EDITED nodes. It also describes tree-transformations that simplify the disfluency detection task by providing alternative encodings of disfluencies and syntactic information.