2020/10/23 by Michael A. Hedderich, Hedderich, Michael A., Lukas Lange +7 · 13 citations
Computer Science · #Topic Modeling #Multimodal Machine Learning Applications #Machine Learning and Algorithms
paper · pdf · doi:10.48550/arxiv.2010.12309
Deep neural networks and huge language models are becoming omnipresent in\nnatural language applications. As they are known for requiring large amounts of\ntraining data, there is a growing body of work to improve the performance in\nlow-resource settings. Motivated by the recent fundamental changes towards\nneural models and the popular pre-train and fine-tune paradigm, we survey\npromising approaches for low-resource natural language processing. After a\ndiscussion about the different dimensions of data availability, we give a\nstructured overview of methods that enable learning when training data is\nsparse. This includes mechanisms to create additional labeled data like data\naugmentation and distant supervision as well as transfer learning settings that\nreduce the need for target supervision. A goal of our survey is to explain how\nthese methods differ in their requirements as understanding them is essential\nfor choosing a technique suited for a specific low-resource setting. Further\nkey aspects of this work are to highlight open issues and to outline promising\ndirections for future research.\n