2018/04/06 by Krzysztof Choromański, Choromanski, Krzysztof, Mark Rowland +7 · 7 citations
Computer Science · #Advanced Neural Network Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics #Robotics (cs.RO) #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.1804.02395
openalex publication_date 2018/04/06 · openalex created_date 2022/09/02 · openalex updated_date 2026/07/28
We present a new method of blackbox optimization via gradient approximation\nwith the use of structured random orthogonal matrices, providing more accurate\nestimators than baselines and with provable theoretical guarantees. We show\nthat this algorithm can be successfully applied to learn better quality compact\npolicies than those using standard gradient estimation techniques. The compact\npolicies we learn have several advantages over unstructured ones, including\nfaster training algorithms and faster inference. These benefits are important\nwhen the policy is deployed on real hardware with limited resources. Further,\ncompact policies provide more scalable architectures for derivative-free\noptimization (DFO) in high-dimensional spaces. We show that most robotics tasks\nfrom the OpenAI Gym can be solved using neural networks with less than 300\nparameters, with almost linear time complexity of the inference phase, with up\nto 13x fewer parameters relative to the Evolution Strategies (ES) algorithm\nintroduced by Salimans et al. (2017). We do not need heuristics such as fitness\nshaping to learn good quality policies, resulting in a simple and theoretically\nmotivated training mechanism.\n