Learning Geometric Word Meta-Embeddings

2020/04/20 by Jawanpuria, Pratik, Dev, N T V Satya, Kunchukuttan, Anoop +1
#Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.2004.09219

Abstract

We propose a geometric framework for learning meta-embeddings of words from different embedding sources. Our framework transforms the embeddings into a common latent space, where, for example, simple averaging of different embeddings (of a given word) is more amenable. The proposed latent space arises from two particular geometric transformations - the orthogonal rotations and the Mahalanobis metric scaling. Empirical results on several word similarity and word analogy benchmarks illustrate the efficacy of the proposed framework.

Related

• Frustratingly Easy Meta-Embedding -- Computing Meta-Embeddings by Averaging Source Word Embeddings
• Learning Meta Word Embeddings by Unsupervised Weighted Concatenation of Source Embeddings
• Learning Meta-Embeddings by Using Ensembles of Embedding Sets
• Think Globally, Embed Locally --- Locally Linear Meta-embedding of Words
• Angular-Based Word Meta-Embedding Learning