2020/05/15 by Shahin Amiriparian, Pawel Winokurow, Amiriparian, Shahin +9
Arts and Humanities · Computer Science · Psychology · #Audio and Speech Processing (eess.AS) #Emotion and Mood Recognition #FOS: Computer and information sciences #FOS: Electrical engineering #Human Pose and Action Recognition #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Sleep and Work-Related Fatigue #Sound (cs.SD) #Subtitles and Audiovisual Media #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2005.08722
openalex publication_date 2020/05/15 · openalex created_date 2022/07/26 · openalex updated_date 2026/07/28
Motivated by the attention mechanism of the human visual system and recent\ndevelopments in the field of machine translation, we introduce our\nattention-based and recurrent sequence to sequence autoencoders for fully\nunsupervised representation learning from audio files. In particular, we test\nthe efficacy of our novel approach on the task of speech-based sleepiness\nrecognition. We evaluate the learnt representations from both autoencoders, and\nthen conduct an early fusion to ascertain possible complementarity between\nthem. In our frameworks, we first extract Mel-spectrograms from raw audio\nfiles. Second, we train recurrent autoencoders on these spectrograms which are\nconsidered as time-dependent frequency vectors. Afterwards, we extract the\nactivations of specific fully connected layers of the autoencoders which\nrepresent the learnt features of spectrograms for the corresponding audio\ninstances. Finally, we train support vector regressors on these representations\nto obtain the predictions. On the development partition of the data, we achieve\nSpearman's correlation coefficients of .324, .283, and .320 with the targets on\nthe Karolinska Sleepiness Scale by utilising attention and non-attention\nautoencoders, and the fusion of both autoencoders' representations,\nrespectively. In the same order, we achieve .311, .359, and .367 Spearman's\ncorrelation coefficients on the test data, indicating the suitability of our\nproposed fusion strategy.\n