Investigating the Impact of Word Informativeness on Speech Emotion Recognition

2025/06/02 by Sofoklis Kakouros, Kakouros, Sofoklis
Psychology · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #Emotion and Mood Recognition #FOS: Computer and information sciences #FOS: Electrical engineering #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2506.02239

openalex publication_date 2025/06/02 · openalex created_date 2025/10/14 · openalex updated_date 2026/07/28

Abstract

In emotion recognition from speech, a key challenge lies in identifying speech signal segments that carry the most relevant acoustic variations for discerning specific emotions. Traditional approaches compute functionals for features such as energy and F0 over entire sentences or longer speech portions, potentially missing essential fine-grained variation in the long-form statistics. This research investigates the use of word informativeness, derived from a pre-trained language model, to identify semantically important segments. Acoustic features are then computed exclusively for these identified segments, enhancing emotion recognition accuracy. The methodology utilizes standard acoustic prosodic features, their functionals, and self-supervised representations. Results indicate a notable improvement in recognition performance when features are computed on segments selected based on word informativeness, underscoring the effectiveness of this approach.

Citations

• Investigating the Utility of Surprisal from Large Language Models for Speech Synthesis Prosody
• Speech-based emotion recognition with self-supervised models using attentive channel-wise correlations and label smoothing
• Extracting speaker and emotion information from self-supervised speech models via channel-wise correlations
• WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
• HuBERT: Self-Supervised Speech Representation Learning by Masked\n Prediction of Hidden Units
• SUPERB: Speech processing Universal PERformance Benchmark
• Emotion Recognition from Speech Using Wav2vec 2.0 Embeddings
• wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
• Emotional speech recognition: Resources, features, and methods

Related

• How Speech is Recognized to Be Emotional - A Study Based on Information Decomposition
• Layer-Wise Analysis of Self-Supervised Acoustic Word Embeddings: A Study on Speech Emotion Recognition
• Describing emotions with acoustic property prompts for speech emotion recognition
• On the Impact of Word Error Rate on Acoustic-Linguistic Speech Emotion Recognition: An Update for the Deep Learning Era
• Leveraging Content and Acoustic Representations for Speech Emotion Recognition