2021/09/09 by Harish Tayyar Madabushi, Madabushi, Harish Tayyar, Edward Gow-Smith +5 · 1 citation
Biochemistry, Genetics and Molecular Biology · Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning in Bioinformatics #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2109.04413
openalex publication_date 2021/09/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Despite their success in a variety of NLP tasks, pre-trained language models,\ndue to their heavy reliance on compositionality, fail in effectively capturing\nthe meanings of multiword expressions (MWEs), especially idioms. Therefore,\ndatasets and methods to improve the representation of MWEs are urgently needed.\nExisting datasets are limited to providing the degree of idiomaticity of\nexpressions along with the literal and, where applicable, (a single)\nnon-literal interpretation of MWEs. This work presents a novel dataset of\nnaturally occurring sentences containing MWEs manually classified into a\nfine-grained set of meanings, spanning both English and Portuguese. We use this\ndataset in two tasks designed to test i) a language model's ability to detect\nidiom usage, and ii) the effectiveness of a language model in generating\nrepresentations of sentences containing idioms. Our experiments demonstrate\nthat, on the task of detecting idiomatic usage, these models perform reasonably\nwell in the one-shot and few-shot scenarios, but that there is significant\nscope for improvement in the zero-shot scenario. On the task of representing\nidiomaticity, we find that pre-training is not always effective, while\nfine-tuning could provide a sample efficient method of learning representations\nof sentences containing MWEs.\n