2019/12/14 by Nicolas Garneau, Garneau, Nicolas, Jean-Samuel Leboeuf +5
Computer Science · #Natural Language Processing Techniques #Topic Modeling #Multimodal Machine Learning Applications
paper · pdf · doi:10.48550/arxiv.1912.06876
We propose a new contextual-compositional neural network layer that handles\nout-of-vocabulary (OOV) words in natural language processing (NLP) tagging\ntasks. This layer consists of a model that attends to both the character\nsequence and the context in which the OOV words appear. We show that our model\nlearns to generate task-specific \and sentence-dependent OOV word\nrepresentations without the need for pre-training on an embedding table, unlike\nprevious attempts. We insert our layer in the state-of-the-art tagging model of\n citetplank2016multilingual and thoroughly evaluate its contribution on 23\ndifferent languages on the task of jointly tagging part-of-speech and\nmorphosyntactic attributes. Our OOV handling method successfully improves\nperformances of this model on every language but one to achieve a new\nstate-of-the-art on the Universal Dependencies Dataset 1.4.\n