Dynamic Neural Network Architectural and Topological Adaptation and Related Methods -- A Survey

2021/07/28 by Lorenz Kummer, Kummer, Lorenz
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.2108.10066

12 pages, preprint

arxiv created 2021/07/28 · openalex publication_date 2021/07/28 · arxiv updated 2021/08/24 · openalex created_date 2021/08/30 · openalex updated_date 2026/07/28

Abstract

Training and inference in deep neural networks (DNNs) has, due to a steady increase in architectural complexity and data set size, lead to the development of strategies for reducing time and space requirements of DNN training and inference, which is of particular importance in scenarios where training takes place in resource constrained computation environments or inference is part of a time critical application. In this survey, we aim to provide a general overview and categorization of state-of-the-art (SOTA) of techniques to reduced DNN training and inference time and space complexities with a particular focus on architectural adaptions.

Related

• GradNets: Dynamic Interpolation Between Neural Architectures
• Leveraging The Topological Consistencies of Learning in Deep Neural Networks
• Differentiable Neural Architecture Search with Morphism-based Transformable Backbone Architectures
• S2DNAS:Transforming Static CNN Model for Dynamic Inference via Neural Architecture Search
• Network Architecture Search for Domain Adaptation