2021/03/11 by Trisha Mittal, Mittal, Trisha, Puneet Mathur +5 · 1 citation
Computer Science · Psychology · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Emotion and Mood Recognition #FOS: Computer and information sciences #Multimedia (cs.MM) #Sentiment Analysis and Opinion Mining
paper · pdf · doi:10.48550/arxiv.2103.06541
openalex publication_date 2021/03/11 · openalex created_date 2022/10/01 · openalex updated_date 2026/07/28
We present Affect2MM, a learning method for time-series emotion prediction\nfor multimedia content. Our goal is to automatically capture the varying\nemotions depicted by characters in real-life human-centric situations and\nbehaviors. We use the ideas from emotion causation theories to computationally\nmodel and determine the emotional state evoked in clips of movies. Affect2MM\nexplicitly models the temporal causality using attention-based methods and\nGranger causality. We use a variety of components like facial features of\nactors involved, scene understanding, visual aesthetics, action/situation\ndescription, and movie script to obtain an affective-rich representation to\nunderstand and perceive the scene. We use an LSTM-based learning model for\nemotion perception. To evaluate our method, we analyze and compare our\nperformance on three datasets, SENDv1, MovieGraphs, and the LIRIS-ACCEDE\ndataset, and observe an average of 10-15% increase in the performance over SOTA\nmethods for all three datasets.\n