2024/05/01 by Dongyuan Li, Ying Zhang, Li, Dongyuan +7
Computer Science · #Artificial Intelligence (cs.AI) #Audio and Speech Processing (eess.AS) #Context-Aware Activity Recognition Systems #FOS: Computer and information sciences #FOS: Electrical engineering #Fuzzy Logic and Control Systems #Sound (cs.SD) #Speech and dialogue systems #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2405.00307
openalex publication_date 2024/05/01 · openalex created_date 2024/05/04 · openalex updated_date 2026/07/28
Speech emotion recognition (SER) has garnered increasing attention due to its wide range of applications in various fields, including human-machine interaction, virtual assistants, and mental health assistance. However, existing SER methods often overlook the information gap between the pre-training speech recognition task and the downstream SER task, resulting in sub-optimal performance. Moreover, current methods require much time for fine-tuning on each specific speech dataset, such as IEMOCAP, which limits their effectiveness in real-world scenarios with large-scale noisy data. To address these issues, we propose an active learning (AL)-based fine-tuning framework for SER, called After, that leverages task adaptation pre-training (TAPT) and AL methods to enhance performance and efficiency. Specifically, we first use TAPT to minimize the information gap between the pre-training speech recognition task and the downstream speech emotion recognition task. Then, AL methods are employed to iteratively select a subset of the most informative and diverse samples for fine-tuning, thereby reducing time consumption. Experiments demonstrate that our proposed method After, using only 20% of samples, improves accuracy by 8.45% and reduces time consumption by 79%. The additional extension of After and ablation studies further confirm its effectiveness and applicability to various real-world scenarios. Our source code is available on Github for reproducibility. (https://github.com/Clearloveyuan/AFTER).