2020/08/08 by Aritz D. Martinez, Javier Del Ser, Martinez, Aritz D. +13 · 1 citation
Biochemistry, Genetics and Molecular Biology · Computer Science · Materials Science · #Artificial Intelligence (cs.AI) #Cell Image Analysis Techniques #Evolutionary Algorithms and Applications #FOS: Computer and information sciences #Machine Learning in Materials Science #Multiagent Systems (cs.MA) #Neural and Evolutionary Computing (cs.NE)
paper · pdf · doi:10.48550/arxiv.2008.03620
openalex publication_date 2020/08/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Much has been said about the fusion of bio-inspired optimization algorithms\nand Deep Learning models for several purposes: from the discovery of network\ntopologies and hyper-parametric configurations with improved performance for a\ngiven task, to the optimization of the model's parameters as a replacement for\ngradient-based solvers. Indeed, the literature is rich in proposals showcasing\nthe application of assorted nature-inspired approaches for these tasks. In this\nwork we comprehensively review and critically examine contributions made so far\nbased on three axes, each addressing a fundamental question in this research\navenue: a) optimization and taxonomy (Why?), including a historical\nperspective, definitions of optimization problems in Deep Learning, and a\ntaxonomy associated with an in-depth analysis of the literature, b) critical\nmethodological analysis (How?), which together with two case studies, allows us\nto address learned lessons and recommendations for good practices following the\nanalysis of the literature, and c) challenges and new directions of research\n(What can be done, and what for?). In summary, three axes - optimization and\ntaxonomy, critical analysis, and challenges - which outline a complete vision\nof a merger of two technologies drawing up an exciting future for this area of\nfusion research.\n