GoEmotions: A Dataset of Fine-Grained Emotions
2020/05/01 by Dorottya Demszky, Demszky, Dorottya, Dana Movshovitz-Attias +9 · 100 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #cs.CL
paper · pdf · doi:10.48550/arxiv.2005.00547
Accepted to ACL 2020
arxiv created 2020/06/03 · arxiv updated 2020/06/04
Abstract
Understanding emotion expressed in language has a wide range of applications, from building empathetic chatbots to detecting harmful online behavior. Advancement in this area can be improved using large-scale datasets with a fine-grained typology, adaptable to multiple downstream tasks. We introduce GoEmotions, the largest manually annotated dataset of 58k English Reddit comments, labeled for 27 emotion categories or Neutral. We demonstrate the high quality of the annotations via Principal Preserved Component Analysis. We conduct transfer learning experiments with existing emotion benchmarks to show that our dataset generalizes well to other domains and different emotion taxonomies. Our BERT-based model achieves an average F1-score of .46 across our proposed taxonomy, leaving much room for improvement.
Cited by
- NepEMO: A Multi-Label Emotion and Sentiment Analysis on Nepali Reddit with Linguistic Insights and Temporal Trends
- Hierarchical Geometry of Cognitive States in Transformer Embedding Spaces
- What do Reward Models Memorize?
- GEMCo: A Validated, Ethically Releasable Proxy for Inaccessible Counselling Data
- Efficient Text Classification with Conformal In-Context Learning
- E3AD: An Emotion-Aware Vision-Language-Action Model for Human-Centric End-to-End Autonomous Driving
- Empirical Prompt Engineering for Construct Identification with Large Language Models
- Story2MIDI: Emotionally Aligned Music Generation from Text
- A Customer Journey in the Land of Oz: Leveraging the Wizard of Oz Technique to Model Emotions in Customer Service Interactions
- MoodBench 1.0: An Evaluation Benchmark for Emotional Companionship Dialogue Systems
- Quality-Controlled Multimodal Emotion Recognition in Conversations with Identity-Based Transfer Learning and MAMBA Fusion
- Based on Data Balancing and Model Improvement for Multi-Label Sentiment Classification Performance Enhancement
- Classification of Hope in Textual Data using Transformer-Based Models
- T-FIX: Text-Based Explanations with Features Interpretable to eXperts
- Structurally Separated Uncertainty in Supervised Latent Variable Models
- Large Emotional World Model
- TOPol: Capturing and Explaining Multidimensional Semantic Polarity Fields and Vectors
- Small Language Models Offer Significant Potential for Science Community
- Emotion-Coherent Reasoning for Multimodal LLMs via Emotional Rationale Verifier
- Emotions Where Art Thou: Understanding and Characterizing the Emotional Latent Space of Large Language Models
- CLiVR: Conversational Learning System in Virtual Reality with AI-Powered Patients
- MARCUS: An Event-Centric NLP Pipeline that generates Character Arcs from Narratives
- How News Feels: Understanding Affective Bias in Multilingual Headlines for Human-Centered Media Design
- Taking a SEAT: Predicting Value Interpretations from Sentiment, Emotion, Argument, and Topic Annotations
- Meta-Harness: End-to-End Optimization of Model Harnesses
- GREAT: Generalizable Backdoor Attacks in RLHF via Emotion-Aware Trigger Synthesis
- From Noise to Signal to Selbstzweck: Reframing Human Label Variation in the Era of Post-training in NLP
- TWIST: Training-free and Label-free Short Text Clustering through Iterative Vector Updating with LLMs
- Fine-Grained Emotion Recognition via In-Context Learning
- Decoding Emotion in the Deep: A Systematic Study of How LLMs Represent, Retain, and Express Emotion
- Implementing federated learning for privacy-preserving emotion detection in educational environments
- Semantic F1 Scores: Fair Evaluation Under Fuzzy Class Boundaries
- Same Facts, Different Diagnosis: Measuring and Mitigating Narrative Anchoring in Clinical Language Models
- EmoHeal: An End-to-End System for Personalized Therapeutic Music Retrieval from Fine-grained Emotions
- SINAI at eRisk@CLEF 2022: Approaching Early Detection of Gambling and Eating Disorders with Natural Language Processing
- A Multi-Component AI Framework for Computational Psychology: From Robust Predictive Modeling to Deployed Generative Dialogue
- The Impact of Annotator Personas on LLM Behavior Across the Perspectivism Spectrum
- Fluent but Unfeeling: The Emotional Blind Spots of Language Models
- Being Kind Isn't Always Being Safe: Diagnosing Affective Hallucination in LLMs
- Dream to Chat: Model-based Reinforcement Learning on Dialogues with User Belief Modeling
- Verbalized Algorithms: Classical Algorithms are All You Need (Mostly)
- From Joy to Fear: A Benchmark of Emotion Estimation in Pop Song Lyrics
- MVRS: The Multimodal Virtual Reality Stimuli-based Emotion Recognition Dataset
- AI in Mental Health: Emotional and Sentiment Analysis of Large Language Models' Responses to Depression, Anxiety, and Stress Queries
- Decoding Neural Emotion Patterns through Large Language Model Embeddings
- Large Language Models for Subjective Language Understanding: A Survey
- Towards Safer AI Moderation: Evaluating LLM Moderators Through a Unified Benchmark Dataset and Advocating a Human-First Approach
- Improving Fine-Grained Emotion Detection in Text with BERT and GoEmotions
- EICAP: Deep Dive in Assessment and Enhancement of Large Language Models in Emotional Intelligence through Multi-Turn Conversations
- AffectGPT-R1: Leveraging Reinforcement Learning for Open-Vocabulary Multimodal Emotion Recognition
- Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoders
- A novel approach to mapping place using a Holocaust survivor testimony
- Backtranslation and paraphrasing in the LLM era? Comparing data augmentation methods for emotion classification
- Moodifier: MLLM-Enhanced Emotion-Driven Image Editing
- Analyzing Emotions in Bangla Social Media Comments Using Machine Learning and LIME
- Emergence of Hierarchical Emotion Organization in Large Language Models
- Co-DETECT: Collaborative Discovery of Edge Cases in Text Classification
- Why We Feel What We Feel: Joint Detection of Emotions and Their Opinion Triggers in E-commerce
- Investigating Algorithmic Bias in YouTube Shorts
- Memory Mosaics at scale
- ARF-RLHF: Adaptive Reward-Following for RLHF through Emotion-Driven Self-Supervision and Trace-Biased Dynamic Optimization
- Knowing Your Users by Heart: A Critical Examination of the Scientific Research on Emotions Conducted by Social Media Platforms
- Can Reasoning Help Large Language Models Capture Human Annotator Disagreement?
- EmoNews: A Spoken Dialogue System for Expressive News Conversations
- PrefixMemory-Tuning: Modernizing Prefix-Tuning by Decoupling the Prefix from Attention
- A Novel, Human-in-the-Loop Computational Grounded Theory Framework for Big Social Data
- Examining the sentiment and emotional differences in product and service reviews: The moderating role of culture
- Think Like a Person Before Responding: A Multi-Faceted Evaluation of Persona-Guided LLMs for Countering Hate
- Crowd-SFT: Crowdsourcing for LLM Alignment
- Abstract Counterfactuals for Language Model Agents
- The Reader is the Metric: How Textual Features and Reader Profiles Explain Conflicting Evaluations of AI Creative Writing
- Statement-Tuning Enables Efficient Cross-lingual Generalization in Encoder-only Models
- CHEER-Ekman: Fine-grained Embodied Emotion Classification
- AnnaAgent: Dynamic Evolution Agent System with Multi-Session Memory for Realistic Seeker Simulation
- SATA-BENCH: Select All That Apply Benchmark for Multiple Choice Questions
- An evaluation of LLMs for generating movie reviews: GPT-4o, Gemini-2.0 and DeepSeek-V3
- MOLE: Metadata Extraction and Validation in Scientific Papers Using LLMs
- Contrastive Distillation of Emotion Knowledge from LLMs for Zero-Shot Emotion Recognition
- Large Language Models Do Multi-Label Classification Differently
- Humans Hallucinate Too: Language Models Identify and Correct Subjective Annotation Errors With Label-in-a-Haystack Prompts
- MAPLE: Many-Shot Adaptive Pseudo-Labeling for In-Context Learning
- MIKU-PAL: An Automated and Standardized Multi-Modal Method for Speech Paralinguistic and Affect Labeling
- AdUE: Improving uncertainty estimation head for LoRA adapters in LLMs
- The Super Emotion Dataset
- Contextual Paralinguistic Data Creation for Multi-Modal Speech-LLM: Data Condensation and Spoken QA Generation
- Co-Designing a Chatbot for Culturally Competent Clinical Communication: Experience and Reflections
- EmoMeta: A Multimodal Dataset for Fine-grained Emotion Classification in Chinese Metaphors
- No Query, No Access
- Examining gender and cultural influences on customer emotions
- Valence-Arousal Subspace in LLMs: Circular Emotion Geometry and Multi-Behavioral Control
- Cross-National Information Attacks: A Two-Decade Analysis of Troll Behavior in Korea
- Sentiment and Social Signals in the Climate Crisis: A Survey on Analyzing Social Media Responses to Extreme Weather Events
- Hype and Adoption of Generative Artificial Intelligence Applications
- Who Gets Which Message? Auditing Demographic Bias in LLM-Generated Targeted Text
- VIBE: A VAD-Informed Benchmark for Entity-Centered Affective Profiling of Large Language Model Outputs
- Can Third-parties Read Our Emotions?
- PicPersona-TOD : A Dataset for Personalizing Utterance Style in Task-Oriented Dialogue with Image Persona
- EEmo-Bench: A Benchmark for Multi-modal Large Language Models on Image Evoked Emotion Assessment
- Emo Pillars: Knowledge Distillation to Support Fine-Grained Context-Aware and Context-Less Emotion Classification
- AI with Emotions: Exploring Emotional Expressions in Large Language Models
Related