Self-Supervised Attention Networks and Uncertainty Loss Weighting for Multi-Task Emotion Recognition on Vocal Bursts

2022/09/15 by Vincent Karas, Andreas Triantafyllopoulos, Karas, Vincent +5
Computer Science · #Artificial Intelligence (cs.AI) #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Music and Audio Processing #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2209.07384

openalex publication_date 2022/09/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Vocal bursts play an important role in communicating affect, making them valuable for improving speech emotion recognition. Here, we present our approach for classifying vocal bursts and predicting their emotional significance in the ACII Affective Vocal Burst Workshop & Challenge 2022 (A-VB). We use a large self-supervised audio model as shared feature extractor and compare multiple architectures built on classifier chains and attention networks, combined with uncertainty loss weighting strategies. Our approach surpasses the challenge baseline by a wide margin on all four tasks.

Related

• Self-Relation Attention and Temporal Awareness for Emotion Recognition via Vocal Burst
• A Hierarchical Regression Chain Framework for Affective Vocal Burst Recognition
• Fine-tuning Wav2vec for Vocal-burst Emotion Recognition
• An Efficient Multitask Learning Architecture for Affective Vocal Burst Analysis
• Attention-Augmented End-to-End Multi-Task Learning for Emotion Prediction from Speech