2022/11/01 by Kenzo Clauw, Sebastiano Stramaglia, Clauw, Kenzo +3 · 1 citation
Biochemistry, Genetics and Molecular Biology · Computer Science · Neuroscience · #FOS: Biological sciences #FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning in Bioinformatics #Neural Networks and Applications #Neural dynamics and brain function #Neurons and Cognition (q-bio.NC)
paper · pdf · doi:10.48550/arxiv.2211.00416
openalex publication_date 2022/11/01 · openalex created_date 2022/11/07 · openalex updated_date 2026/07/28
Quantifying which neurons are important with respect to the classification decision of a trained neural network is essential for understanding their inner workings. Previous work primarily attributed importance to individual neurons. In this work, we study which groups of neurons contain synergistic or redundant information using a multivariate mutual information method called the O-information. We observe the first layer is dominated by redundancy suggesting general shared features (i.e. detecting edges) while the last layer is dominated by synergy indicating local class-specific features (i.e. concepts). Finally, we show the O-information can be used for multi-neuron importance. This can be demonstrated by re-training a synergistic sub-network, which results in a minimal change in performance. These results suggest our method can be used for pruning and unsupervised representation learning.