2016/03/07 by Yangfeng Ji, Ji, Yangfeng, Gholamreza Haffari +3 · 4 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Natural Language Processing Techniques #Neural and Evolutionary Computing (cs.NE) #Speech and dialogue systems #Topic Modeling
paper · pdf · doi:10.48550/arxiv.1603.01913
openalex publication_date 2016/03/07 · openalex created_date 2016/06/24 · openalex updated_date 2026/07/28
This paper presents a novel latent variable recurrent neural network architecture for jointly modeling sequences of words and (possibly latent) discourse relations between adjacent sentences. A recurrent neural network generates individual words, thus reaping the benefits of discriminatively-trained vector representations. The discourse relations are represented with a latent variable, which can be predicted or marginalized, depending on the task. The resulting model can therefore employ a training objective that includes not only discourse relation classification, but also word prediction. As a result, it outperforms state-of-the-art alternatives for two tasks: implicit discourse relation classification in the Penn Discourse Treebank, and dialog act classification in the Switchboard corpus. Furthermore, by marginalizing over latent discourse relations at test time, we obtain a discourse informed language model, which improves over a strong LSTM baseline.