A Baseline Multimodal Approach to Emotion Recognition in Conversations
2026/01/31 by Víctor Yeste, Rodrigo Rivas-Arévalo · 1 voice
Computer Science · Engineering · #cs.AI #cs.CL #cs.CY #cs.SD #eess.AS
paper · pdf · doi:10.48550/arxiv.2602.00914
Abstract
We present a lightweight multimodal baseline for emotion recognition in conversations using the SemEval-2024 Task 3 dataset built from the sitcom Friends. The goal of this report is not to propose a novel state-of-the-art method, but to document an accessible reference implementation that combines (i) a transformer-based text classifier and (ii) a self-supervised speech representation model, with a simple late-fusion ensemble. We report the baseline setup and empirical results obtained under a limited training protocol, highlighting when multimodal fusion improves over unimodal models. This preprint is provided for transparency and to support future, more rigorous comparisons.
Citations
- Efficient Multimodal Large Language Models: A Survey
- A Comprehensive Survey on Affective Computing; Challenges, Trends, Applications, and Future Directions
- Do you know "saudade"? The importance of a cultural and language-based emotion approach for HCI
- Classification of Emotions and Evaluation of Customer Satisfaction from\n Speech in Real World Acoustic Environments
- Emotion Dynamics Modeling via BERT
- Multi-Modal Emotion Detection with Transfer Learning
- wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
- DeBERTa: Decoding-enhanced BERT with Disentangled Attention
- DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
- RoBERTa: A Robustly Optimized BERT Pretraining Approach
- Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances
- Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling
- Emotion Detection in Text: a Review
- A review of affective computing: From unimodal analysis to multimodal fusion
- Tensor Fusion Network for Multimodal Sentiment Analysis
- Attention Is All You Need
- Multimodal Machine Learning: A Survey and Taxonomy
- Multimodal Machine Learning: A Survey and Taxonomy
- A review of depression and suicide risk assessment using speech analysis
- Emotional speech recognition: Resources, features, and methods
Discussions
Related