Dynamic Compositionality in Recursive Neural Networks with Structure-aware Tag Representations

2018/09/07 by Taeuk Kim, Kim, Taeuk, Jihun Choi +7
Biochemistry, Genetics and Molecular Biology · Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning in Bioinformatics #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1809.02286

openalex publication_date 2018/09/07 · openalex created_date 2019/06/27 · openalex updated_date 2026/07/28

Abstract

Most existing recursive neural network (RvNN) architectures utilize only the structure of parse trees, ignoring syntactic tags which are provided as by-products of parsing. We present a novel RvNN architecture that can provide dynamic compositionality by considering comprehensive syntactic information derived from both the structure and linguistic tags. Specifically, we introduce a structure-aware tag representation constructed by a separate tag-level tree-LSTM. With this, we can control the composition function of the existing word-level tree-LSTM by augmenting the representation as a supplementary input to the gate functions of the tree-LSTM. In extensive experiments, we show that models built upon the proposed architecture obtain superior or competitive performance on several sentence-level tasks such as sentiment analysis and natural language inference when compared against previous tree-structured models and other sophisticated neural models.

Citations

• ADADELTA: An Adaptive Learning Rate Method
• Adam: A Method for Stochastic Optimization
• Learning to Compose Words into Sentences with Reinforcement Learning
• Learning to Generate Reviews and Discovering Sentiment
• Learned in Translation: Contextualized Word Vectors
• Shortcut-Stacked Sentence Encoders for Multi-Domain Inference
• Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales
• A large annotated corpus for learning natural language inference

Related

• Learning to Compose over Tree Structures via POS Tags
• Joint RNN-Based Greedy Parsing and Word Composition
• Tree-structured composition in neural networks without tree-structured architectures
• Dynamic Compositional Neural Networks over Tree Structure
• Unsupervised Learning of Explainable Parse Trees for Improved Generalisation