2020/01/02 by Michael Neunert, Abbas Abdolmaleki, Neunert, Michael +17 · 3 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · #FOS: Computer and information sciences #Formal Methods in Verification #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics #Robotics (cs.RO) #Viral Infectious Diseases and Gene Expression in Insects
paper · pdf · doi:10.48550/arxiv.2001.00449
openalex publication_date 2020/01/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Many real-world control problems involve both discrete decision variables -\nsuch as the choice of control modes, gear switching or digital outputs - as\nwell as continuous decision variables - such as velocity setpoints, control\ngains or analogue outputs. However, when defining the corresponding optimal\ncontrol or reinforcement learning problem, it is commonly approximated with\nfully continuous or fully discrete action spaces. These simplifications aim at\ntailoring the problem to a particular algorithm or solver which may only\nsupport one type of action space. Alternatively, expert heuristics are used to\nremove discrete actions from an otherwise continuous space. In contrast, we\npropose to treat hybrid problems in their 'native' form by solving them with\nhybrid reinforcement learning, which optimizes for discrete and continuous\nactions simultaneously. In our experiments, we first demonstrate that the\nproposed approach efficiently solves such natively hybrid reinforcement\nlearning problems. We then show, both in simulation and on robotic hardware,\nthe benefits of removing possibly imperfect expert-designed heuristics. Lastly,\nhybrid reinforcement learning encourages us to rethink problem definitions. We\npropose reformulating control problems, e.g. by adding meta actions, to improve\nexploration or reduce mechanical wear and tear.\n