Training Deep Networks for Facial Expression Recognition with Crowd-Sourced Label Distribution

2016/08/03 by Emad Barsoum, Barsoum, Emad, Cha Zhang +5 · 17 citations
Computer Science · Psychology · #Emotion and Mood Recognition #Face and Expression Recognition #Music and Audio Processing #cs.CV

paper · pdf · doi:10.48550/arxiv.1608.01041

Submitted to ICMI 2016

arxiv created 2016/09/24 · arxiv updated 2016/09/27

Abstract

Crowd sourcing has become a widely adopted scheme to collect ground truth labels. However, it is a well-known problem that these labels can be very noisy. In this paper, we demonstrate how to learn a deep convolutional neural network (DCNN) from noisy labels, using facial expression recognition as an example. More specifically, we have 10 taggers to label each input image, and compare four different approaches to utilizing the multiple labels: majority voting, multi-label learning, probabilistic label drawing, and cross-entropy loss. We show that the traditional majority voting scheme does not perform as well as the last two approaches that fully leverage the label distribution. An enhanced FER+ data set with multiple labels for each face image will also be shared with the research community.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions

Cited by

• A Perception CNN for Facial Expression Recognition
• Exploring Fusion Strategies for Multimodal Vision-Language Systems
• Global Multiple Extraction Network for Low-Resolution Facial Expression Recognition
• Rethinking Facial Expression Recognition in the Era of Multimodal Large Language Models: Benchmark, Datasets, and Beyond
• Facial-Expression-Aware Prompting for Empathetic LLM Tutoring
• ExpressNet-MoE: A Hybrid Deep Neural Network for Emotion Recognition
• NeuroGaze-Distill: Brain-informed Distillation and Depression-Inspired Geometric Priors for Robust Facial Emotion Recognition
• NeuroDeX: Unlocking Diverse Support in Decompiling Deep Neural Network Executables
• Performance of Conformal Prediction in Capturing Aleatoric Uncertainty
• User Experience Estimation in Human-Robot Interaction Via Multi-Instance Learning of Multimodal Social Signals
• Learning Transferable Facial Emotion Representations from Large-Scale Semantically Rich Captions
• FedMultiEmo: Real-Time Emotion Recognition via Multimodal Federated Learning
• Multimodal Prompt Alignment for Facial Expression Recognition
• MMME: A Spontaneous Multi-Modal Micro-Expression Dataset Enabling Visual-Physiological Fusion
• PixelSmile: Toward Fine-Grained Facial Expression Editing
• An Evaluation of a Visual Question Answering Strategy for Zero-shot Facial Expression Recognition in Still Images
• Evaluating Test-Time Adaptation For Facial Expression Recognition Under Natural Cross-Dataset Distribution Shifts

Related

• ReSup: Reliable Label Noise Suppression for Facial Expression Recognition
• Training Deep Neural Networks on Noisy Labels with Bootstrapping
• A Light CNN for Deep Face Representation with Noisy Labels
• Uncertainty-aware Label Distribution Learning for Facial Expression Recognition
• Efficient Facial Feature Learning with Wide Ensemble-based Convolutional Neural Networks