SenWiCh: Sense-Annotation of Low-Resource Languages for WiC using Hybrid Methods

2025/05/29 by Roksana Goworek, Harpal Singh Karlcut, Goworek, Roksana +23 · 1 citation
Computer Science · Social Sciences · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #ICT in Developing Communities #Multilingual Education and Policy #Natural Language Processing Techniques

paper · pdf · doi:10.48550/arxiv.2505.23714

openalex publication_date 2025/05/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper addresses the critical need for high-quality evaluation datasets in low-resource languages to advance cross-lingual transfer. While cross-lingual transfer offers a key strategy for leveraging multilingual pretraining to expand language technologies to understudied and typologically diverse languages, its effectiveness is dependent on quality and suitable benchmarks. We release new sense-annotated datasets of sentences containing polysemous words, spanning ten low-resource languages across diverse language families and scripts. To facilitate dataset creation, the paper presents a demonstrably beneficial semi-automatic annotation method. The utility of the datasets is demonstrated through Word-in-Context (WiC) formatted experiments that evaluate transfer on these low-resource languages. Results highlight the importance of targeted dataset creation and evaluation for effective polysemy disambiguation in low-resource settings and transfer studies. The released datasets and code aim to support further research into fair, robust, and truly multilingual NLP.

Citations

• Multilingual Instruction Tuning With Just a Pinch of Multilinguality
• Towards a Common Understanding of Contributing Factors for Cross-Lingual Transfer in Multilingual Language Models: A Review
• MEGA: Multilingual Evaluation of Generative AI
• L3Cube-HindBERT and DevBERT: Pre-Trained BERT Transformer models for Devanagari based Hindi and Marathi Languages
• Predicting the Performance of Multilingual NLP Models
• Towards a Unified View of Parameter-Efficient Transfer Learning
• Finetuned Language Models Are Zero-Shot Learners
• Analysing The Impact Of Linguistic Features On Cross-Lingual Transfer
• BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?
• AM2iCo: Evaluating Word Meaning in Context across Low-Resource Languages with Adversarial Examples
• Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages
• MuRIL: Multilingual Representations for Indian Languages
• FEWS: Large-Scale, Low-Shot Word Sense Disambiguation with the Dictionary
• XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualization
• WiC-TSV: An Evaluation Benchmark for Target Sense Verification of Words\n in Context
• AI4Bharat-IndicNLP Corpus: Monolingual Corpora and Word Embeddings for Indic Languages
• PhoBERT: Pre-trained language models for Vietnamese
• Unsupervised Cross-lingual Representation Learning at Scale
• How multilingual is Multilingual BERT?
• WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations
• BabelNet: The automatic construction, evaluation and application of a wide-coverage multilingual semantic network
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Cited by

• Multilinguality Does not Make Sense: Investigating Factors Behind Zero-Shot Transfer in Sense-Aware Tasks

Related

• XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualization
• SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects
• Beyond Counting Datasets: A Survey of Multilingual Dataset Construction and Necessary Resources
• Low-Resource, High-Impact: Building Corpora for Inclusive Language Technologies
• Huge Automatically Extracted Training Sets for Multilingual Word Sense Disambiguation