A Simple and Effective Approach for Fine Tuning Pre-trained Word\n Embeddings for Improved Text Classification

2019/08/07 by Amr Al-Khatib, Al-Khatib, Amr, Samhaa R. El-Beltagy +1
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Sentiment Analysis and Opinion Mining #Text and Document Classification Technologies #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1908.02579

openalex publication_date 2019/08/07 · openalex created_date 2022/07/28 · openalex updated_date 2026/07/28

Abstract

This work presents a new and simple approach for fine-tuning pretrained word\nembeddings for text classification tasks. In this approach, the class in which\na term appears, acts as an additional contextual variable during the fine\ntuning process, and contributes to the final word vector for that term. As a\nresult, words that are used distinctively within a particular class, will bear\nvectors that are closer to each other in the embedding space and will be more\ndiscriminative towards that class. To validate this novel approach, it was\napplied to three Arabic and two English datasets that have been previously used\nfor text classification tasks such as sentiment analysis and emotion detection.\nIn the vast majority of cases, the results obtained using the proposed\napproach, improved considerably.\n

Citations

• Supervised Fine Tuning for Word Embedding with Integrated Knowledge
• Long Short-Term Memory
• Distributed Representations of Sentences and Documents
• A Survey on Transfer Learning

Related

• Domain Lexical Knowledge-based Word Embedding Learning for Text Classification under Small Data
• Word Embeddings and Their Use In Sentence Classification Tasks
• Utility of General and Specific Word Embeddings for Classifying Translational Stages of Research
• Good, Better, Best: Choosing Word Embedding Context
• TF-CR: Weighting Embeddings for Text Classification