A Local Optima Network Analysis of the Feedforward Neural Architecture Space

2022/06/02 by Isak Potgieter, Potgieter, Isak, Christopher W. Cleghorn +3
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and ELM #Metaheuristic Optimization Algorithms Research #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE)

paper · pdf · doi:10.48550/arxiv.2206.06903

openalex publication_date 2022/06/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This study investigates the use of local optima network (LON) analysis, a derivative of the fitness landscape of candidate solutions, to characterise and visualise the neural architecture space. The search space of feedforward neural network architectures with up to three layers, each with up to 10 neurons, is fully enumerated by evaluating trained model performance on a selection of data sets. Extracted LONs, while heterogeneous across data sets, all exhibit simple global structures, with single global funnels in all cases but one. These results yield early indication that LONs may provide a viable paradigm by which to analyse and optimise neural architectures.

Related

• Loss Surface Modality of Feed-Forward Neural Network Architectures
• Structure of Artificial Neural Networks -- Empirical Investigations
• Optimisation & Generalisation in Networks of Neurons
• Redefining Neural Architecture Search of Heterogeneous Multi-Network Models by Characterizing Variation Operators and Model Components
• Making Differentiable Architecture Search less local