Reinforcement Learning for Improving Agent Design

2019/11/01 by David Ha · 4 citations
Computer Science · Engineering · #Multimodal Machine Learning Applications #Reinforcement Learning in Robotics #Robot Manipulation and Learning

paper · pdf · doi:10.1162/artl_a_00301

openalex publication_date 2019/11/01 · openalex created_date 2019/11/22 · openalex updated_date 2026/07/31

Abstract

In many reinforcement learning tasks, the goal is to learn a policy to manipulate an agent, whose design is fixed, to maximize some notion of cumulative reward. The design of the agent's physical structure is rarely optimized for the task at hand. In this work, we explore the possibility of learning a version of the agent's design that is better suited for its task, jointly with the policy. We propose an alteration to the popular OpenAI Gym framework, where we parameterize parts of an environment, and allow an agent to jointly learn to modify these environment parameters along with its policy. We demonstrate that an agent can learn a better structure of its body that is not only better suited for the task, but also facilitates policy learning. Joint learning of policy and structure may even uncover design principles that are useful for assisted-design applications.

Cited by

• The Emergence of Complex Behavior in Large-Scale Ecological Environments
• A Design Co-Pilot for Task-Tailored Manipulators
• Embodied intelligence via learning and evolution
• VLMgineer: Vision Language Models as Robotic Toolsmiths

Related

• An agent design with goal reaching guarantees for enhancement of learning
• Structural Design Through Reinforcement Learning
• Policy Fusion for Adaptive and Customizable Reinforcement Learning Agents
• Iterative Policy-Space Expansion in Reinforcement Learning
• Policy composition in reinforcement learning via multi-objective policy optimization