Emotions are Subtle: Learning Sentiment Based Text Representations Using Contrastive Learning

2021/12/02 by Ipsita Mohanty, Ankit Goyal, Mohanty, Ipsita +3
Computer Science · Psychology · Social Sciences · #Computation and Language (cs.CL) #Computational and Text Analysis Methods #FOS: Computer and information sciences #Mental Health via Writing #Sentiment Analysis and Opinion Mining #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2112.01054

openalex publication_date 2021/12/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Contrastive learning techniques have been widely used in the field of computer vision as a means of augmenting datasets. In this paper, we extend the use of these contrastive learning embeddings to sentiment analysis tasks and demonstrate that fine-tuning on these embeddings provides an improvement over fine-tuning on BERT-based embeddings to achieve higher benchmarks on the task of sentiment analysis when evaluated on the DynaSent dataset. We also explore how our fine-tuned models perform on cross-domain benchmark datasets. Additionally, we explore upsampling techniques to achieve a more balanced class distribution to make further improvements on our benchmark tasks.

Related

• Bridging Visual Affective Gap: Borrowing Textual Knowledge by Learning from Noisy Image-Text Pairs
• Mere Contrastive Learning for Cross-Domain Sentiment Analysis
• Retrofitting Light-weight Language Models for Emotions using Supervised Contrastive Learning
• Continuous Adversarial Text Representation Learning for Affective Recognition
• CLMLF:A Contrastive Learning and Multi-Layer Fusion Method for Multimodal Sentiment Detection