Variable Impedance Control in End-Effector Space: An Action Space for\n Reinforcement Learning in Contact-Rich Tasks

2019/06/20 by Roberto Martín-Martín, Michelle A. Lee, Martín-Martín, Roberto +9 · 11 citations
Engineering · Neuroscience · #Robot Manipulation and Learning #Muscle activation and electromyography studies #Motor Control and Adaptation

paper · pdf · doi:10.48550/arxiv.1906.08880

Abstract

Reinforcement Learning (RL) of contact-rich manipulation tasks has yielded\nimpressive results in recent years. While many studies in RL focus on varying\nthe observation space or reward model, few efforts focused on the choice of\naction space (e.g. joint or end-effector space, position, velocity, etc.).\nHowever, studies in robot motion control indicate that choosing an action space\nthat conforms to the characteristics of the task can simplify exploration and\nimprove robustness to disturbances. This paper studies the effect of different\naction spaces in deep RL and advocates for Variable Impedance Control in\nEnd-effector Space (VICES) as an advantageous action space for constrained and\ncontact-rich tasks. We evaluate multiple action spaces on three prototypical\nmanipulation tasks: Path Following (task with no contact), Door Opening (task\nwith kinematic constraints), and Surface Wiping (task with continuous contact).\nWe show that VICES improves sample efficiency, maintains low energy\nconsumption, and ensures safety across all three experimental setups. Further,\nRL policies learned with VICES can transfer across different robot models in\nsimulation, and from simulation to real for the same robot. Further information\nis available at https://stanfordvl.github.io/vices.\n

Cited by

• WARL: Wrench-Augmented Reinforcement Learning for Task-Agnostic Learning in Legged Robots
• Kinematics-Aware Diffusion Policy with Consistent 3D Observation and Action Space for Whole-Arm Robotic Manipulation
• CHIP: Adaptive Compliance for Humanoid Control through Hindsight Perturbation
• Invariance Co-training for Robot Visual Generalization
• ContactRL: Safe Reinforcement Learning based Motion Planning for Contact based Human Robot Collaboration
• Variable-Impedance Muscle Coordination under Slow-Rate Control Frequencies and Limited Observation Conditions Evaluated through Legged Locomotion
• HAFO: A Force-Adaptive Control Framework for Humanoid Robots in Intense Interaction Environments
• Flow with the Force Field: Learning 3D Compliant Flow Matching Policies from Force and Demonstration-Guided Simulation Data
• SHaRe-RL: Structured, Interactive Reinforcement Learning for Contact-Rich Industrial Assembly Tasks
• Impedance Primitive-augmented Hierarchical Reinforcement Learning for Sequential Tasks
• Multimodal Visual Transformer for Sim2real Transfer in Visual Reinforcement Learning

Related

• Learning Variable Impedance Control via Inverse Reinforcement Learning for Force-Related Tasks
• Learning Variable Impedance Control for Contact Sensitive Tasks
• Impedance Adaptation by Reinforcement Learning with Contact Dynamic Movement Primitives
• A Comparison of Action Spaces for Learning Manipulation Tasks
• Impedance Primitive-augmented Hierarchical Reinforcement Learning for Sequential Tasks