2016/08/29 by Junqi Jin, Ziang Yan, Jin, Junqi +7 · 1 citation
Computer Science · Mathematics · #Advanced Neural Network Applications #Algorithm #Artificial intelligence #Artificial neural network #Budget constraint #Computer science #Constraint (computer-aided design) #FOS: Computer and information sciences #Greedy algorithm #Key (lock) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Mathematical optimization #Mathematics #Recurrent neural network #Selection (genetic algorithm) #Submodular set function #Topic Modeling #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.1608.07892
published in arXiv (Cornell University) (Cornell University) · Withdrawn due to incompleteness and some overlaps with existing literatures, I will resubmit adding further results
openalex publication_date 2016/08/29 · openalex created_date 2016/09/16 · arxiv created 2018/02/21 · arxiv updated 2018/02/22 · openalex updated_date 2026/07/28
Recurrent neural network (RNN)'s architecture is a key factor influencing its performance. We propose algorithms to optimize hidden sizes under running time constraint. We convert the discrete optimization into a subset selection problem. By novel transformations, the objective function becomes submodular and constraint becomes supermodular. A greedy algorithm with bounds is suggested to solve the transformed problem. And we show how transformations influence the bounds. To speed up optimization, surrogate functions are proposed which balance exploration and exploitation. Experiments show that our algorithms can find more accurate models or faster models than manually tuned state-of-the-art and random search. We also compare popular RNN architectures using our algorithms.