2018/12/21 by Guoyun Tu, Tu, Guoyun, Yanwei Fu +10 · 1 citation
Computer Science · Mathematics · Psychology · #Artificial intelligence #Artificial neural network #Attribution #Automatic summarization #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Emotion and Mood Recognition #Emotion classification #FOS: Computer and information sciences #Human Pose and Action Recognition #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Natural language processing #Pattern recognition (psychology) #Psychology #Social psychology #Task (project management) #Video Analysis and Summarization #cs.CV #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.1812.09041
Authors' manuscript; published at the IEEE Transactions on Multimedia
openalex publication_date 2018/12/21 · openalex created_date 2019/01/01 · arxiv created 2019/07/24 · arxiv updated 2019/07/25 · openalex updated_date 2026/08/05
Emotional content is a crucial ingredient in user-generated videos. However, the sparsity of emotional expressions in the videos poses an obstacle to visual emotion analysis. In this paper, we propose a new neural approach, Bi-stream Emotion Attribution-Classification Network (BEAC-Net), to solve three related emotion analysis tasks: emotion recognition, emotion attribution, and emotion-oriented summarization, in a single integrated framework. BEAC-Net has two major constituents, an attribution network and a classification network. The attribution network extracts the main emotional segment that classification should focus on in order to mitigate the sparsity issue. The classification network utilizes both the extracted segment and the original video in a bi-stream architecture. We contribute a new dataset for the emotion attribution task with human-annotated ground-truth labels for emotion segments. Experiments on two video datasets demonstrate superior performance of the proposed framework and the complementary nature of the dual classification streams.