Unsupervised Hyperalignment for Multilingual Word Embeddings

2018/11/02 by Jean Alaux, Édouard Grave, Alaux, Jean +6 · 1 citation
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling #cs.CL #cs.LG

paper · pdf · doi:10.48550/arxiv.1811.01124

ICLR 2019

openalex publication_date 2018/11/02 · arxiv created 2019/06/04 · arxiv updated 2019/06/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We consider the problem of aligning continuous word representations, learned in multiple languages, to a common space. It was recently shown that, in the case of two languages, it is possible to learn such a mapping without supervision. This paper extends this line of work to the problem of aligning multiple languages to a common space. A solution is to independently map all languages to a pivot language. Unfortunately, this degrades the quality of indirect word translation. We thus propose a novel formulation that ensures composable mappings, leading to better alignments. We evaluate our method by jointly aligning word vectors in eleven languages, showing consistent improvement with indirect mappings while maintaining competitive performance on direct word translation.

Citations

• A Generalized Solution of the Orthogonal Procrustes Problem
• Exploiting Similarities among Languages for Machine Translation
• Distributed Representations of Words and Phrases and their Compositionality
• Word Translation Without Parallel Data
• Unsupervised Alignment of Embeddings with Wasserstein Procrustes
• Unsupervised Multilingual Word Embeddings
• Offline bilingual word vectors, orthogonal transformations and the inverted softmax
• Enriching Word Vectors with Subword Information

Cited by

• Linear-Time Gromov Wasserstein Distances using Low Rank Couplings and\n Costs

Related

• A Simple Approach to Learning Unsupervised Multilingual Embeddings
• Word Alignment by Fine-tuning Embeddings on Parallel Corpora
• Trans-gram, Fast Cross-lingual Word-embeddings
• Unsupervised Word Translation Pairing using Refinement based Point Set Registration
• SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings