Classifying the classifier: dissecting the weight space of neural\n networks

2020/02/13 by Gabriel Eilertsen, Daniel Jönsson, Eilertsen, Gabriel +7 · 10 citations
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Artificial intelligence #Artificial neural network #Classifier (UML) #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Deep neural networks #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification #Machine learning #Neural Networks and Applications #Pattern recognition (psychology) #Space (punctuation) #cs.CV #cs.LG

paper · pdf · doi:10.48550/arxiv.2002.05688

published in arXiv (Cornell University), 1119-1126 (Cornell University) · ECAI 2020

arxiv created 2020/02/13 · openalex publication_date 2020/02/13 · arxiv updated 2020/02/14 · openalex created_date 2022/07/26 · openalex updated_date 2026/08/04

Abstract

This paper presents an empirical study on the weights of neural networks,\nwhere we interpret each model as a point in a high-dimensional space -- the\nneural weight space. To explore the complex structure of this space, we sample\nfrom a diverse selection of training variations (dataset, optimization\nprocedure, architecture, etc.) of neural network classifiers, and train a large\nnumber of models to represent the weight space. Then, we use a machine learning\napproach for analyzing and extracting information from this space. Most\ncentrally, we train a number of novel deep meta-classifiers with the objective\nof classifying different properties of the training setup by identifying their\nfootprints in the weight space. Thus, the meta-classifiers probe for patterns\ninduced by hyper-parameters, so that we can quantify how much, where, and when\nthese are encoded through the optimization process. This provides a novel and\ncomplementary view for explainable AI, and we show how meta-classifiers can\nreveal a great deal of information about the training setup and optimization,\nby only considering a small subset of randomly selected consecutive weights. To\npromote further research on the weight space, we release the neural weight\nspace (NWS) dataset -- a collection of 320K weight snapshots from 16K\nindividually trained deep neural networks.\n

Citations

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Understanding Neural Networks Through Deep Visualization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Gradient-based learning applied to document recognition
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability
• Sensitivity and Generalization in Neural Networks: an Empirical Study
• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
• DARTS: Differentiable Architecture Search
• Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• Learning Multiple Layers of Features from Tiny Images

Cited by

• Metanetworks as Regulatory Operators: Learning to Edit for Requirement Compliance
• Learning Model Representations Using Publicly Available Model Hubs
• Why is topology hard to learn?
• An open dataset of neural networks for hypernetwork research
• GradMetaNet: An Equivariant Architecture for Learning on Gradients
• Data Swarms: Optimizable Generation of Synthetic Evaluation Data
• Discovering Hidden Gems in Model Repositories
• A Model Zoo on Phase Transitions in Neural Networks
• A Model Zoo of Vision Transformers
• The Impact of Model Zoo Size and Composition on Weight Space Learning

Related

• MLDS: A Dataset for Weight-Space Analysis of Neural Networks
• Model Zoo Dataset Samples for Scalable Weight Space Learning
• Hyper-Representations: Learning from Populations of Neural Networks
• Some Insights into the Geometry and Training of Neural Networks
• Improving classifier decision boundaries using nearest neighbors