Fusing Vector Space Models for Domain-Specific Applications

2019/09/05 by Laura Rettig, Rettig, Laura, Julien Audiffren +3
Computer Science · Mathematics · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #cs.CL #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1909.02307

ICTAI 2019

arxiv created 2019/09/05 · arxiv updated 2019/09/06

Abstract

We address the problem of tuning word embeddings for specific use cases and domains. We propose a new method that automatically combines multiple domain-specific embeddings, selected from a wide range of pre-trained domain-specific embeddings, to improve their combined expressive power. Our approach relies on two key components: 1) a ranking function, based on a new embedding similarity measure, that selects the most relevant embeddings to use given a domain and 2) a dimensionality reduction method that combines the selected embeddings to produce a more compact and efficient encoding that preserves the expressiveness. We empirically show that our method produces effective domain-specific embeddings that consistently improve the performance of state-of-the-art machine learning algorithms on multiple tasks, compared to generic embeddings trained on large text corpora.

Related

• An Ensemble Method to Produce High-Quality Word Embeddings (2016)
• Single Training Dimension Selection for Word Embedding with PCA
• Repurposing Language Models into Embedding Models: Finding the Compute-Optimal Recipe
• Compressing Word Embeddings
• Learning Word Embeddings with Domain Awareness