Joint Learning of Pre-Trained and Random Units for Domain Adaptation in\n Part-of-Speech Tagging

2019/04/07 by Sara Meftah, Meftah, Sara, Youssef Tamaazousti +7
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1904.03595

openalex publication_date 2019/04/07 · openalex created_date 2022/07/29 · openalex updated_date 2026/07/28

Abstract

Fine-tuning neural networks is widely used to transfer valuable knowledge\nfrom high-resource to low-resource domains. In a standard fine-tuning scheme,\nsource and target problems are trained using the same architecture. Although\ncapable of adapting to new domains, pre-trained units struggle with learning\nuncommon target-specific patterns. In this paper, we propose to augment the\ntarget-network with normalised, weighted and randomly initialised units that\nbeget a better adaptation while maintaining the valuable source knowledge. Our\nexperiments on POS tagging of social media texts (Tweets domain) demonstrate\nthat our method achieves state-of-the-art performances on 3 commonly used\ndatasets.\n

Related

• Domain adaptation for part-of-speech tagging of noisy user-generated\n text
• Neural Supervised Domain Adaptation by Augmenting Pre-trained Models with Random Units
• Unsupervised Domain Adaptation using Lexical Transformations and Label Injection for Twitter Data
• R2T: Rule-Encoded Loss Functions for Low-Resource Sequence Tagging
• Domain Adaptation for Neural Networks by Parameter Augmentation