2020/02/25 by Jitin Krishnan, Krishnan, Jitin, Hemant Purohit +3
Computer Science · #Computation and Language (cs.CL) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)
paper · pdf · doi:10.48550/arxiv.2002.10937
openalex publication_date 2020/02/25 · openalex created_date 2022/07/26 · openalex updated_date 2026/07/28
Domain adaptation approaches seek to learn from a source domain and\ngeneralize it to an unseen target domain. At present, the state-of-the-art\nunsupervised domain adaptation approaches for subjective text classification\nproblems leverage unlabeled target data along with labeled source data. In this\npaper, we propose a novel method for domain adaptation of single-task text\nclassification problems based on a simple but effective idea of diversity-based\ngeneralization that does not require unlabeled target data but still matches\nthe state-of-the-art in performance. Diversity plays the role of promoting the\nmodel to better generalize and be indiscriminate towards domain shift by\nforcing the model not to rely on same features for prediction. We apply this\nconcept on the most explainable component of neural networks, the attention\nlayer. To generate sufficient diversity, we create a multi-head attention model\nand infuse a diversity constraint between the attention heads such that each\nhead will learn differently. We further expand upon our model by tri-training\nand designing a procedure with an additional diversity constraint between the\nattention heads of the tri-trained classifiers. Extensive evaluation using the\nstandard benchmark dataset of Amazon reviews and a newly constructed dataset of\nCrisis events shows that our fully unsupervised method matches with the\ncompeting baselines that uses unlabeled target data. Our results demonstrate\nthat machine learning architectures that ensure sufficient diversity can\ngeneralize better; encouraging future research to design ubiquitously usable\nlearning models without using unlabeled target data.\n