EPNAS: Efficient Progressive Neural Architecture Search

2019/07/07 by Yanqi Zhou, Peng Wang, Zhou, Yanqi +11
Computer Science · #Advanced Neural Network Applications #Anomaly Detection Techniques and Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Algorithms

paper · pdf · doi:10.48550/arxiv.1907.04648

openalex publication_date 2019/07/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we propose Efficient Progressive Neural Architecture Search (EPNAS), a neural architecture search (NAS) that efficiently handles large search space through a novel progressive search policy with performance prediction based on REINFORCE~\citeWilliams.1992.PG. EPNAS is designed to search target networks in parallel, which is more scalable on parallel systems such as GPU/TPU clusters. More importantly, EPNAS can be generalized to architecture search with multiple resource constraints, \eg, model size, compute complexity or intensity, which is crucial for deployment in widespread platforms such as mobile and cloud. We compare EPNAS against other state-of-the-art (SoTA) network architectures (\eg, MobileNetV2~\citemobilenetv2) and efficient NAS algorithms (\eg, ENAS~\citepham2018efficient, and PNAS~\citeLiu2017b) on image recognition tasks using CIFAR10 and ImageNet. On both datasets, EPNAS is superior \wrt architecture searching speed and recognition accuracy.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Learning both Weights and Connections for Efficient Neural Networks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Neural Architecture Search with Reinforcement Learning
• Large-Scale Evolution of Image Classifiers
• DeepArchitect: Automatically Designing and Training Deep Architectures
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• DARTS: Differentiable Architecture Search
• SNAS: Stochastic Neural Architecture Search
• Xception: Deep Learning with Depthwise Separable Convolutions
• Efficient Architecture Search by Network Transformation
• Learning Multiple Layers of Features from Tiny Images

Related

• Efficient Progressive Neural Architecture Search
• PONAS: Progressive One-shot Neural Architecture Search for Very Efficient Deployment
• Progressive DARTS: Bridging the Optimization Gap for NAS in the Wild
• DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search
• EENA: Efficient Evolution of Neural Architecture