Attending to Emotional Narratives

2019/07/08 by Zhengxuan Wu, Xiyu Zhang, Wu, Zhengxuan +7 · 1 citation
Computer Science · Psychology · #Computation and Language (cs.CL) #Emotion and Mood Recognition #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Sentiment Analysis and Opinion Mining #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1907.04197

openalex publication_date 2019/07/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Attention mechanisms in deep neural networks have achieved excellent performance on sequence-prediction tasks. Here, we show that these recently-proposed attention-based mechanisms---in particular, the Transformer with its parallelizable self-attention layers, and the Memory Fusion Network with attention across modalities and time---also generalize well to multimodal time-series emotion recognition. Using a recently-introduced dataset of emotional autobiographical narratives, we adapt and apply these two attention mechanisms to predict emotional valence over time. Our models perform extremely well, in some cases reaching a performance comparable with human raters. We end with a discussion of the implications of attention mechanisms to affective computing.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Empathy: Its ultimate and proximate bases
• A Concordance Correlation Coefficient to Evaluate Reproducibility
• Memory Fusion Network for Multi-view Sequential Learning

Cited by

• Leveraging Vision Transformers for Enhanced Classification of Emotions using ECG Signals

Related

• A Self-Attentive Emotion Recognition Network
• Temporal Aware Mixed Attention-based Convolution and Transformer Network (MACTN) for EEG Emotion Recognition
• Attention-based Region of Interest (ROI) Detection for Speech Emotion Recognition
• Real-Time Emotion Recognition via Attention Gated Hierarchical Memory Network
• Multi-modal Conditional Attention Fusion for Dimensional Emotion Prediction