2021/08/05 by Sarala Padi, Seyed Omid Sadjadi, Padi, Sarala +5
Computer Science · Psychology · #Speech and Audio Processing #Emotion and Mood Recognition #Speech Recognition and Synthesis
paper · pdf · doi:10.48550/arxiv.2108.02510
Automatic speech emotion recognition (SER) is a challenging task that plays a\ncrucial role in natural human-computer interaction. One of the main challenges\nin SER is data scarcity, i.e., insufficient amounts of carefully labeled data\nto build and fully explore complex deep learning models for emotion\nclassification. This paper aims to address this challenge using a transfer\nlearning strategy combined with spectrogram augmentation. Specifically, we\npropose a transfer learning approach that leverages a pre-trained residual\nnetwork (ResNet) model including a statistics pooling layer from speaker\nrecognition trained using large amounts of speaker-labeled data. The statistics\npooling layer enables the model to efficiently process variable-length input,\nthereby eliminating the need for sequence truncation which is commonly used in\nSER systems. In addition, we adopt a spectrogram augmentation technique to\ngenerate additional training data samples by applying random time-frequency\nmasks to log-mel spectrograms to mitigate overfitting and improve the\ngeneralization of emotion recognition models. We evaluate the effectiveness of\nour proposed approach on the interactive emotional dyadic motion capture\n(IEMOCAP) dataset. Experimental results indicate that the transfer learning and\nspectrogram augmentation approaches improve the SER performance, and when\ncombined achieve state-of-the-art results.\n