Dependency Parsing with LSTMs: An Empirical Evaluation

2016/04/22 by Kuncoro, Adhiguna, Sawai, Yuichiro, Duh, Kevin +1
#Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural and Evolutionary Computing (cs.NE)

paper · doi:10.48550/arxiv.1604.06529

Abstract

We propose a transition-based dependency parser using Recurrent Neural Networks with Long Short-Term Memory (LSTM) units. This extends the feedforward neural network parser of Chen and Manning (2014) and enables modelling of entire sequences of shift/reduce transition decisions. On the Google Web Treebank, our LSTM parser is competitive with the best feedforward parser on overall accuracy and notably achieves more than 3% improvement for long-range dependencies, which has proved difficult for previous transition-based parsers due to error propagation and limited context information. Our findings additionally suggest that dropout regularisation on the embedding layer is crucial to improve the LSTM's generalisation.

Related

• A Simple LSTM model for Transition-based Dependency Parsing
• Transition-Based Dependency Parsing with Stack Long Short-Term Memory
• Recursive Subtree Composition in LSTM-Based Dependency Parsing
• Improved Transition-Based Parsing by Modeling Characters instead of Words with LSTMs
• Transition-Based Dependency Parsing using Perceptron Learner