2021/08/04 by Anubhav Bhatti, Behnam Behinaein, Bhatti, Anubhav +7
Computer Science · Engineering · Psychology · #Emotion and Mood Recognition #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Signal Processing (eess.SP) #cs.LG #eess.SP #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2108.02241
5 pages, 2 figures. Accepted at 2021 9th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)
arxiv created 2021/08/04 · openalex publication_date 2021/08/04 · arxiv updated 2021/08/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Classification of human emotions can play an essential role in the design and improvement of human-machine systems. While individual biological signals such as Electrocardiogram (ECG) and Electrodermal Activity (EDA) have been widely used for emotion recognition with machine learning methods, multimodal approaches generally fuse extracted features or final classification/regression results to boost performance. To enhance multimodal learning, we present a novel attentive cross-modal connection to share information between convolutional neural networks responsible for learning individual modalities. Specifically, these connections improve emotion classification by sharing intermediate representations among EDA and ECG and apply attention weights to the shared information, thus learning more effective multimodal embeddings. We perform experiments on the WESAD dataset to identify the best configuration of the proposed method for emotion classification. Our experiments show that the proposed approach is capable of learning strong multimodal representations and outperforms a number of baselines methods.