Understanding trained CNNs by indexing neuron selectivity

2017/02/28 by Ivet Rafegas, María Vanrell, Maria Vanrell +3
Biochemistry, Genetics and Molecular Biology · Computer Science · Engineering · Neuroscience · Psychology · #Artificial intelligence #Cell Image Analysis Techniques #Class (philosophy) #Computer science #Convolutional neural network #Feature (linguistics) #Image Processing Techniques and Applications #Neural Networks and Applications #Neuron #Neuroscience #Pattern recognition (psychology) #Psychology #Search engine indexing #Visualization #cs.CV

paper · pdf · doi:10.1016/j.patrec.2019.10.013

Under review on Pattern Recognition Letters

arxiv created 2019/05/21 · openalex publication_date 2019/10/15 · arxiv updated 2019/10/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

The impressive performance of Convolutional Neural Networks (CNNs) when solving different vision problems is shadowed by their black-box nature and our consequent lack of understanding of the representations they build and how these representations are organized. To help understanding these issues, we propose to describe the activity of individual neurons by their Neuron Feature visualization and quantify their inherent selectivity with two specific properties. We explore selectivity indexes for: an image feature (color); and an image label (class membership). Our contribution is a framework to seek or classify neurons by indexing on these selectivity properties. It helps to find color selective neurons, such as a red-mushroom neuron in layer Conv4 or class selective neurons such as dog-face neurons in layer Conv5 in VGG-M, and establishes a methodology to derive other selectivity properties. Indexing on neuron selectivity can statistically draw how features and classes are represented through layers in a moment when the size of trained nets is growing and automatic tools to index neurons can be helpful.

Citations

• Why does Deep Learning work? - A perspective from Group Theory
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Understanding Neural Networks Through Deep Visualization
• Explaining and Harnessing Adversarial Examples
• Return of the Devil in the Details: Delving Deep into Convolutional Nets
• Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks
• Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
• Distilling a Neural Network Into a Soft Decision Tree
• Revisiting the Importance of Individual Units in CNNs via Ablation
• MatConvNet - Convolutional Neural Networks for MATLAB
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps