Rethinking Feature Distribution for Loss Functions in Image Classification

2018/03/08 by Weitao Wan, Yuanyi Zhong, Wan, Weitao +5 · 3 citations
Computer Science · Medicine · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #COVID-19 diagnosis using AI #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · pdf · doi:10.48550/arxiv.1803.02988

openalex publication_date 2018/03/08 · openalex created_date 2018/03/29 · openalex updated_date 2026/07/28

Abstract

We propose a large-margin Gaussian Mixture (L-GM) loss for deep neural networks in classification tasks. Different from the softmax cross-entropy loss, our proposal is established on the assumption that the deep features of the training set follow a Gaussian Mixture distribution. By involving a classification margin and a likelihood regularization, the L-GM loss facilitates both a high classification performance and an accurate modeling of the training feature distribution. As such, the L-GM loss is superior to the softmax loss and its major variants in the sense that besides classification, it can be readily used to distinguish abnormal inputs, such as the adversarial examples, based on their features' likelihood to the training feature distribution. Extensive experiments on various recognition benchmarks like MNIST, CIFAR, ImageNet and LFW, as well as on adversarial examples demonstrate the effectiveness of our proposal.

Citations

• Learning Face Representation from Scratch
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Distilling the Knowledge in a Neural Network
• Improving neural networks by preventing co-adaptation of feature detectors
• Gradient-based learning applied to document recognition
• Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups
• Wide Residual Networks
• FractalNet: Ultra-Deep Neural Networks without Residuals
• Densely Connected Convolutional Networks
• Large-Margin Softmax Loss for Convolutional Neural Networks
• Noisy Softmax: Improving the Generalization Ability of DCNN via Postponing the Early Softmax Saturation
• Deep Residual Learning for Image Recognition
• Deep Learning Face Attributes in the Wild
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Understanding deep learning requires rethinking generalization
• Deeply learned face representations are sparse, selective, and robust
• Learning Multiple Layers of Features from Tiny Images

Cited by

• Person Recognition in Aerial Surveillance: A Decade Survey
• Feature Space Topology Control via Hopkins Loss
• Kaleidoscopic Background Attack: Disrupting Pose Estimation with Multi-Fold Radial Symmetry Textures

Related

• Shaping Deep Feature Space towards Gaussian Mixture for Visual Classification
• Large-Margin Softmax Loss for Convolutional Neural Networks
• Large Margin Deep Networks for Classification
• Negative Log Likelihood Ratio Loss for Deep Neural Network Classification
• Competing Ratio Loss for Discriminative Multi-class Image Classification