2018/01/04 by Charles Schaff, Schaff, Charles, David Yunis +5 · 10 citations
Engineering · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Robotic Locomotion and Control #Robotics (cs.RO)
paper · pdf · doi:10.48550/arxiv.1801.01432
openalex publication_date 2018/01/04 · openalex created_date 2022/10/01 · openalex updated_date 2026/07/28
The physical design of a robot and the policy that controls its motion are\ninherently coupled, and should be determined according to the task and\nenvironment. In an increasing number of applications, data-driven and\nlearning-based approaches, such as deep reinforcement learning, have proven\neffective at designing control policies. For most tasks, the only way to\nevaluate a physical design with respect to such control policies is\nempirical--i.e., by picking a design and training a control policy for it.\nSince training these policies is time-consuming, it is computationally\ninfeasible to train separate policies for all possible designs as a means to\nidentify the best one. In this work, we address this limitation by introducing\na method that performs simultaneous joint optimization of the physical design\nand control network. Our approach maintains a distribution over designs and\nuses reinforcement learning to optimize a control policy to maximize expected\nreward over the design distribution. We give the controller access to design\nparameters to allow it to tailor its policy to each design in the distribution.\nThroughout training, we shift the distribution towards higher-performing\ndesigns, eventually converging to a design and control policy that are jointly\noptimal. We evaluate our approach in the context of legged locomotion, and\ndemonstrate that it discovers novel designs and walking gaits, outperforming\nbaselines in both performance and efficiency.\n