Predicting Neural Network Accuracy from Weights

2020/02/26 by Thomas Unterthiner, Unterthiner, Thomas, Daniel Keysers +7 · 10 citations
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2002.11448

openalex publication_date 2020/02/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We show experimentally that the accuracy of a trained neural network can be predicted surprisingly well by looking only at its weights, without evaluating it on input data. We motivate this task and introduce a formal setting for it. Even when using simple statistics of the weights, the predictors are able to rank neural networks by their performance with very high accuracy (R2 score more than 0.98). Furthermore, the predictors are able to rank networks trained on different, unobserved datasets and with different architectures. We release a collection of 120k convolutional neural networks trained on four different datasets to encourage further research in this area, with the goal of understanding network training and performance better.

Citations

• Learning Multiple Layers of Features from Tiny Images

Cited by

• Metanetworks as Regulatory Operators: Learning to Edit for Requirement Compliance
• Leveraging weights signals -- Predicting and improving generalizability in reinforcement learning
• Symmetry-Aware Graph Metanetwork Autoencoders: Model Merging through Parameter Canonicalization
• Learning Model Representations Using Publicly Available Model Hubs
• Symmetry-Aware Fully-Amortized Optimization with Scale Equivariant Graph Metanetworks
• SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass
• Deep Learning Through the Lens of Example Difficulty
• Understanding and Accelerating Neural Architecture Search with Training-Free and Theory-Grounded Metrics
• Increasing Trustworthiness of Deep Neural Networks via Accuracy Monitoring
• NeuroVoxel-LM: Language-Aligned 3D Perception via Dynamic Voxelization and Meta-Embedding

Related

• Test Sample Accuracy Scales with Training Sample Density in Neural Networks
• Enhancing Accuracy and Parameter-Efficiency of Neural Representations for Network Parameterization
• A comparative analysis of a neural network with calculated weights and a neural network with random generation of weights based on the training dataset size
• Weight Agnostic Neural Networks
• Bit-wise Training of Neural Network Weights