DOTS: Decoupling Operation and Topology in Differentiable Architecture Search

2020/10/02 by Yuchao Gu, Yu-Chao Gu, Gu, Yu-Chao +13 · 2 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · Engineering · Mathematics · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Algorithm #Cell Image Analysis Techniques #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Correctness #Decoupling (probability) #Differentiable function #Discrete mathematics #Engineering #Extension topology #FOS: Computer and information sciences #General topology #Mathematics #Network topology #Topological space #Topology (electrical circuits) #cs.CV

paper · pdf · doi:10.48550/arxiv.2010.00969

openalex publication_date 2020/10/02 · openalex created_date 2020/10/08 · arxiv created 2021/04/08 · arxiv updated 2021/04/09 · openalex updated_date 2026/07/28

Abstract

Differentiable Architecture Search (DARTS) has attracted extensive attention due to its efficiency in searching for cell structures. DARTS mainly focuses on the operation search and derives the cell topology from the operation weights. However, the operation weights can not indicate the importance of cell topology and result in poor topology rating correctness. To tackle this, we propose to Decouple the Operation and Topology Search (DOTS), which decouples the topology representation from operation weights and makes an explicit topology search. DOTS is achieved by introducing a topology search space that contains combinations of candidate edges. The proposed search space directly reflects the search objective and can be easily extended to support a flexible number of edges in the searched cell. Existing gradient-based NAS methods can be incorporated into DOTS for further improvement by the topology search. Considering that some operations (e.g., Skip-Connection) can affect the topology, we propose a group operation search scheme to preserve topology-related operations for a better topology search. The experiments on CIFAR10/100 and ImageNet demonstrate that DOTS is an effective solution for differentiable NAS.

Citations

• A NEW MEASURE OF RANK CORRELATION
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• DARTS: Differentiable Architecture Search
• Single Path One-Shot Neural Architecture Search with Uniform Sampling
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• DARTS+: Improved Differentiable Architecture Search with Early Stopping
• Blockwisely Supervised Neural Architecture Search with Knowledge Distillation
• PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search
• Understanding Architectures Learnt by Cell-based Neural Architecture Search
• DropNAS: Grouped Operation Dropout for Differentiable Architecture Search
• DARTS-: Robustly Stepping out of Performance Collapse Without Indicators
• Learning Multiple Layers of Features from Tiny Images
• Hierarchical Neural Architecture Search via Operator Clustering

Cited by

• DNAD: Differentiable Neural Architecture Distillation
• FX-DARTS: Designing Topology-unconstrained Architectures with Differentiable Architecture Search and Entropy-based Super-network Shrinking

Related

• Unchain the Search Space with Hierarchical Differentiable Architecture Search
• FX-DARTS: Designing Topology-unconstrained Architectures with Differentiable Architecture Search and Entropy-based Super-network Shrinking
• D-DARTS: Distributed Differentiable Architecture Search
• Stretchable Cells Help DARTS Search Better
• Towards Improving the Consistency, Efficiency, and Flexibility of Differentiable Neural Architecture Search