2021/11/01 by Parul Chopra, Sai Krishna Rallabandi, Chopra, Parul +5 · 1 citation
Computer Science · Psychology · Social Sciences · #Speech and dialogue systems #Language Development and Disorders #Multilingual Education and Policy
paper · pdf · doi:10.48550/arxiv.2111.01231
Code-switching (CS), a ubiquitous phenomenon due to the ease of communication\nit offers in multilingual communities still remains an understudied problem in\nlanguage processing. The primary reasons behind this are: (1) minimal efforts\nin leveraging large pretrained multilingual models, and (2) the lack of\nannotated data. The distinguishing case of low performance of multilingual\nmodels in CS is the intra-sentence mixing of languages leading to switch\npoints. We first benchmark two sequence labeling tasks -- POS and NER on 4\ndifferent language pairs with a suite of pretrained models to identify the\nproblems and select the best performing model, char-BERT, among them\n(addressing (1)). We then propose a self training method to repurpose the\nexisting pretrained models using a switch-point bias by leveraging unannotated\ndata (addressing (2)). We finally demonstrate that our approach performs well\non both tasks by reducing the gap between the switch point performance while\nretaining the overall performance on two distinct language pairs in both the\ntasks. Our code is available here:\nhttps://github.com/PC09/EMNLP2021-Switch-Point-biased-Self-Training.\n