2020/04/30 by Andrey Kutuzov, Mario Giulianelli, Kutuzov, Andrey +1 · 3 citations
Computer Science · Physics and Astronomy · #Advanced Text Analysis Techniques #Complex Network Analysis Techniques #Computation and Language (cs.CL) #FOS: Computer and information sciences #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2005.00050
openalex publication_date 2020/04/30 · openalex created_date 2022/10/01 · openalex updated_date 2026/07/28
We apply contextualised word embeddings to lexical semantic change detection\nin the SemEval-2020 Shared Task 1. This paper focuses on Subtask 2, ranking\nwords by the degree of their semantic drift over time. We analyse the\nperformance of two contextualising architectures (BERT and ELMo) and three\nchange detection algorithms. We find that the most effective algorithms rely on\nthe cosine similarity between averaged token embeddings and the pairwise\ndistances between token embeddings. They outperform strong baselines by a large\nmargin (in the post-evaluation phase, we have the best Subtask 2 submission for\nSemEval-2020 Task 1), but interestingly, the choice of a particular algorithm\ndepends on the distribution of gold scores in the test set.\n