Hierarchical Representations for Efficient Architecture Search

2017/11/01 by Hanxiao Liu, Karen Simonyan, Liu, Hanxiao +7 · 2 voices · 10 citations
Computer Science · #Evolutionary Algorithms and Applications #Machine Learning and Data Classification #Metaheuristic Optimization Algorithms Research #cs.CV #cs.LG #cs.NE #stat.ML

paper · pdf · doi:10.48550/arxiv.1711.00436

openalex publication_date 2017/11/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We explore efficient neural architecture search methods and show that a simple yet powerful evolutionary algorithm can discover new architectures with excellent performance. Our approach combines a novel hierarchical genetic representation scheme that imitates the modularized design pattern commonly adopted by human experts, and an expressive search space that supports complex topologies. Our algorithm efficiently discovers architectures that outperform a large number of manually designed models for image classification, obtaining top-1 error of 3.6% on CIFAR-10 and 20.3% when transferred to ImageNet, which is competitive with the best existing neural architecture search approaches. We also present results using random search, achieving 0.3% less top-1 accuracy on CIFAR-10 and 0.1% less on ImageNet whilst reducing the search time from 36 hours down to 1 hour.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• ImageNet Large Scale Visual Recognition Challenge
• Wide Residual Networks
• FractalNet: Ultra-Deep Neural Networks without Residuals
• Densely Connected Convolutional Networks
• Neural Architecture Search with Reinforcement Learning
• SMASH: One-Shot Model Architecture Search through HyperNetworks
• Deep Residual Learning for Image Recognition
• Xception: Deep Learning with Depthwise Separable Convolutions

Cited by

• Spiking Neural Network Architecture Search: A Survey
• ONNX-Net: Towards Universal Representations and Instant Performance Prediction for Neural Architectures
• Accelerating Deep Learning Inference with Cross-Layer Data Reuse on GPUs
• Edge-featured Graph Neural Architecture Search
• GPNAS: A Neural Network Architecture Search Framework Based on Graphical Predictor
• FDNAS: Improving Data Privacy and Model Diversity in AutoML
• Sampled Training and Node Inheritance for Fast Evolutionary Neural Architecture Search
• MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation
• Efficient Neural Network Compression
• Neural Architecture Dilation for Adversarial Robustness

Discussions

• Hierarchical Representations for Efficient Architecture Search (2017) [hn, 45 points, 0 comments]
• HIERARCHICAL REPRESENTATIONS FOR EFFICIENT ARCHITECTURE SEARCH [hn, 2 points, 0 comments]

Related

• A Novel Evolutionary Algorithm for Hierarchical Neural Architecture Search
• Neural Architecture Search using Progressive Evolution
• Enhanced Gradient for Differentiable Architecture Search
• Guided Evolution for Neural Architecture Search
• Guided Evolutionary Neural Architecture Search With Efficient Performance Estimation