Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive\n Autoencoders

2019/04/03 by Andrew Drozdov, Pat Verga, Drozdov, Andrew +7 · 6 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1904.02142

openalex publication_date 2019/04/03 · openalex created_date 2022/07/29 · openalex updated_date 2026/07/28

Abstract

We introduce deep inside-outside recursive autoencoders (DIORA), a\nfully-unsupervised method for discovering syntax that simultaneously learns\nrepresentations for constituents within the induced tree. Our approach predicts\neach word in an input sentence conditioned on the rest of the sentence and uses\ninside-outside dynamic programming to consider all possible binary trees over\nthe sentence. At test time the CKY algorithm extracts the highest scoring\nparse. DIORA achieves a new state-of-the-art F1 in unsupervised binary\nconstituency parsing (unlabeled) in two benchmark datasets, WSJ and MultiNLI.\n

Cited by

• Probability Distribution Collapse: A Critical Bottleneck to Compact Unsupervised Neural Grammar Induction
• Understanding Syntactic Generalization in Structure-inducing Language Models
• What is Learned in Visually Grounded Neural Syntax Acquisition
• StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language Modeling
• Video-aided Unsupervised Grammar Induction
• Montague Grammar Induction

Related

• Improved Latent Tree Induction with Distant Supervision via Span Constraints
• Rule Augmented Unsupervised Constituency Parsing
• Recursive Top-Down Production for Sentence Generation with Latent Trees
• Unsupervised Discontinuous Constituency Parsing with Mildly Context-Sensitive Grammars
• Heads-up! Unsupervised Constituency Parsing via Self-Attention Heads