2018/09/07 by Takashi Wada, Wada, Takashi, Tomoharu Iwata +1
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling
paper · pdf · doi:10.48550/arxiv.1809.02306
openalex publication_date 2018/09/07 · openalex created_date 2022/08/03 · openalex updated_date 2026/07/28
We propose an unsupervised method to obtain cross-lingual embeddings without\nany parallel data or pre-trained word embeddings. The proposed model, which we\ncall multilingual neural language models, takes sentences of multiple languages\nas an input. The proposed model contains bidirectional LSTMs that perform as\nforward and backward language models, and these networks are shared among all\nthe languages. The other parameters, i.e. word embeddings and linear\ntransformation between hidden states and outputs, are specific to each\nlanguage. The shared LSTMs can capture the common sentence structure among all\nlanguages. Accordingly, word embeddings of each language are mapped into a\ncommon latent space, making it possible to measure the similarity of words\nacross multiple languages. We evaluate the quality of the cross-lingual word\nembeddings on a word alignment task. Our experiments demonstrate that our model\ncan obtain cross-lingual embeddings of much higher quality than existing\nunsupervised models when only a small amount of monolingual data (i.e. 50k\nsentences) are available, or the domains of monolingual data are different\nacross languages.\n