Reinforcement Learning for High-dimensional Continuous Control in Biomechanics: An Intro to ArtiSynth-RL

2019/10/25 by Abdi, Amir H., Malakoutian, Masoud, Oxland, Thomas +1
#FOS: Electrical engineering #Signal Processing (eess.SP) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.1910.13859

Abstract

Neural control is an exciting mystery which we instinctively master. Yet, researchers have a hard time explaining the motor control trajectories. Physiologically accurate biomechanical simulations can, to some extent, mimic live subjects and help us form evidence-based hypotheses. In these simulated environments, muscle excitations are typically calculated through inverse dynamic optimizations which do not possess a closed-form solution. Thus, computationally expensive, and occasionally unstable, iterative numerical solvers are the only widely utilized solution. In this work, we introduce ArtiSynth, a 3D modeling platform that supports the combined simulation of multi-body and finite element models, and extended to support reinforcement learning (RL) training. we further use ArtiSynth to investigate whether a deep RL policy can be trained to drive the motor control of a physiologically accurate biomechanical model in a large continuous action space. We run a comprehensive evaluation of its performance and compare the results with the forward dynamics assisted tracking with a quadratic objective function. We assess the two approaches in terms of correctness, stability, energy-efficiency, and temporal consistency.

Related

• Reinforcement Learning of Musculoskeletal Control from Functional Simulations
• Learning to Run challenge: Synthesizing physiologically accurate motion using deep reinforcement learning
• An Efficient Learning Control Framework With Sim-to-Real for String-Type Artificial Muscle-Driven Robotic Systems
• Massively Parallel Imitation Learning of Mouse Forelimb Musculoskeletal Reaching Dynamics
• Neuromuscular Reinforcement Learning to Actuate Human Limbs through FES