2021/10/15 by Sosuke Nishikawa, Ikuya Yamada, Nishikawa, Sosuke +5
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Text and Document Classification Technologies #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2110.07792
openalex publication_date 2021/10/15 · openalex created_date 2021/10/25 · openalex updated_date 2026/07/28
We present a multilingual bag-of-entities model that effectively boosts the performance of zero-shot cross-lingual text classification by extending a multilingual pre-trained language model (e.g., M-BERT). It leverages the multilingual nature of Wikidata: entities in multiple languages representing the same concept are defined with a unique identifier. This enables entities described in multiple languages to be represented using shared embeddings. A model trained on entity features in a resource-rich language can thus be directly applied to other languages. Our experimental results on cross-lingual topic classification (using the MLDoc and TED-CLDC datasets) and entity typing (using the SHINRA2020-ML dataset) show that the proposed model consistently outperforms state-of-the-art models.