2019/11/09 by Yue Cao, Cao, Yue, Tianlong Chen +4 · 1 citation
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #Biomolecules (q-bio.BM) #FOS: Biological sciences #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification #Stochastic Gradient Optimization Techniques
paper · doi:10.48550/arxiv.1911.03787
openalex publication_date 2019/11/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Learning to optimize has emerged as a powerful framework for various optimization and machine learning tasks. Current such "meta-optimizers" often learn in the space of continuous optimization algorithms that are point-based and uncertainty-unaware. To overcome the limitations, we propose a meta-optimizer that learns in the algorithmic space of both point-based and population-based optimization algorithms. The meta-optimizer targets at a meta-loss function consisting of both cumulative regret and entropy. Specifically, we learn and interpret the update formula through a population of LSTMs embedded with sample- and feature-level attentions. Meanwhile, we estimate the posterior directly over the global optimum and use an uncertainty measure to help guide the learning process. Empirical results over non-convex test functions and the protein-docking application demonstrate that this new meta-optimizer outperforms existing competitors. The codes are publicly available at: https://github.com/Shen-Lab/LOIS.