Language Embeddings for Typology and Cross-lingual Transfer Learning

2021/06/03 by Yu, Dian, He, Taiqi, Sagae, Kenji
#Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2106.02082

Abstract

Cross-lingual language tasks typically require a substantial amount of annotated data or parallel translation data. We explore whether language representations that capture relationships among languages can be learned and subsequently leveraged in cross-lingual tasks without the use of parallel data. We generate dense embeddings for 29 languages using a denoising autoencoder, and evaluate the embeddings using the World Atlas of Language Structures (WALS) and two extrinsic tasks in a zero-shot setting: cross-lingual dependency parsing and cross-lingual natural language inference.

Related

• Machine Translation with Cross-lingual Word Embeddings
• Cross-lingual Dependency Parsing with Unlabeled Auxiliary Languages
• Inducing Language-Agnostic Multilingual Representations
• Best Practices for Learning Domain-Specific Cross-Lingual Embeddings
• Emerging Language Spaces Learned From Massively Multilingual Corpora