Facial Affect Recognition based on Multi Architecture Encoder and Feature Fusion for the ABAW7 Challenge

2024/07/17 by Kang Shen, Xuxiong Liu, Shen, Kang +15
Psychology · #Computer Vision and Pattern Recognition (cs.CV) #Emotion and Mood Recognition #FOS: Computer and information sciences

paper · pdf · doi:10.48550/arxiv.2407.12258

openalex publication_date 2024/07/17 · openalex created_date 2024/09/09 · openalex updated_date 2026/07/28

Abstract

In this paper, we present our approach to addressing the challenges of the 7th ABAW competition. The competition comprises three sub-challenges: Valence Arousal (VA) estimation, Expression (Expr) classification, and Action Unit (AU) detection. To tackle these challenges, we employ state-of-the-art models to extract powerful visual features. Subsequently, a Transformer Encoder is utilized to integrate these features for the VA, Expr, and AU sub-challenges. To mitigate the impact of varying feature dimensions, we introduce an affine module to align the features to a common dimension. Overall, our results significantly outperform the baselines.

Related

• Facial Affect Recognition based on Transformer Encoder and Audiovisual Fusion for the ABAW5 Challenge
• ABAW : Facial Expression Recognition in the wild
• Enhancing Facial Expression Recognition through Dual-Direction Attention Mixed Feature Networks and CLIP: Application to 8th ABAW Challenge
• SS-MFAR : Semi-supervised Multi-task Facial Affect Recognition
• Multi-modal Facial Affective Analysis based on Masked Autoencoder