Peephole: Predicting Network Performance Before Training

2017/12/09 by Boyang Deng, Junjie Yan, Deng, Boyang +3 · 3 citations
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Computer Vision and Pattern Recognition (cs.CV) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural and Evolutionary Computing (cs.NE)

paper · pdf · doi:10.48550/arxiv.1712.03351

openalex publication_date 2017/12/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The quest for performant networks has been a significant force that drives the advancements of deep learning in recent years. While rewarding, improving network design has never been an easy journey. The large design space combined with the tremendous cost required for network training poses a major obstacle to this endeavor. In this work, we propose a new approach to this problem, namely, predicting the performance of a network before training, based on its architecture. Specifically, we develop a unified way to encode individual layers into vectors and bring them together to form an integrated description via LSTM. Taking advantage of the recurrent network's strong expressive power, this method can reliably predict the performances of various network architectures. Our empirical studies showed that it not only achieved accurate predictions but also produced consistent rankings across datasets -- a key desideratum in performance prediction.

Citations

• Freeze-Thaw Bayesian Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Long Short-Term Memory
• Gradient-based learning applied to document recognition
• Densely Connected Convolutional Networks
• Neural Architecture Search with Reinforcement Learning
• Deep Residual Learning for Image Recognition
• Practical Block-wise Neural Network Architecture Generation
• Aggregated Residual Transformations for Deep Neural Networks

Cited by

• Graph Lineages and Skeletal Graph Products
• NN-Former: Rethinking Graph Structure in Neural Architecture Representation
• Language Embedding Meets Dynamic Graph: A New Exploration for Neural Architecture Representation Learning

Related

• Parameter Prediction for Unseen Deep Architectures
• Surprisingly Strong Performance Prediction with Neural Graph Features
• Deeper and Wider Networks for Performance Metrics Prediction in Communication Networks
• TART: Token-based Architecture Transformer for Neural Network Performance Prediction
• E-LSTM-D: A Deep Learning Framework for Dynamic Network Link Prediction