AutoBSS: An Efficient Algorithm for Block Stacking Style Search

2020/10/20 by Yikang Zhang, Zhang, Yikang, Zhao Zhong +2 · 1 citation
Computer Science · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2010.10261

openalex publication_date 2020/10/20 · openalex created_date 2020/10/29 · openalex updated_date 2026/07/28

Abstract

Neural network architecture design mostly focuses on the new convolutional operator or special topological structure of network block, little attention is drawn to the configuration of stacking each block, called Block Stacking Style (BSS). Recent studies show that BSS may also have an unneglectable impact on networks, thus we design an efficient algorithm to search it automatically. The proposed method, AutoBSS, is a novel AutoML algorithm based on Bayesian optimization by iteratively refining and clustering Block Stacking Style Code (BSSC), which can find optimal BSS in a few trials without biased evaluation. On ImageNet classification task, ResNet50/MobileNetV2/EfficientNet-B0 with our searched BSS achieve 79.29%/74.5%/77.79%, which outperform the original baselines by a large margin. More importantly, experimental results on model compression, object detection and instance segmentation show the strong generalizability of the proposed AutoBSS, and further verify the unneglectable impact of BSS on neural networks.

Citations

• The Pascal Visual Object Classes (VOC) Challenge
• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
• Gradient-based learning applied to document recognition
• Large-Scale Evolution of Image Classifiers
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• DARTS: Differentiable Architecture Search
• Sigmoid-weighted linear units for neural network function approximation in reinforcement learning
• ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
• Once-for-All: Train One Network and Specialize it for Efficient Deployment
• BETANAS: BalancEd TrAining and selective drop for Neural Architecture Search

Cited by

• One-Shot Neural Architecture Search with Network Similarity Directed Initialization for Pathological Image Classification

Related

• BS-NAS: Broadening-and-Shrinking One-Shot NAS with Searchable Numbers of Channels
• Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
• BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search
• AutoShrink: A Topology-aware NAS for Discovering Efficient Neural Architecture
• Densely Connected Search Space for More Flexible Neural Architecture Search