2023/04/03 by Linhan Yang, Bidan Huang, Yang, Linhan +11 · 8 citations
Engineering · Neuroscience · #EEG and Brain-Computer Interfaces #FOS: Computer and information sciences #Robot Manipulation and Learning #Robotics (cs.RO) #Tactile and Sensory Interactions
paper · pdf · doi:10.48550/arxiv.2304.00736
openalex publication_date 2023/04/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In this paper, we propose a novel framework for tactile-based dexterous manipulation learning with a blind anthropomorphic robotic hand, i.e. without visual sensing. First, object-related states were extracted from the raw tactile signals by a graph-based perception model - TacGNN. The resulting tactile features were then utilized in the policy learning of an in-hand manipulation task in the second stage. This method was examined by a Baoding ball task - simultaneously manipulating two spheres around each other by 180 degrees in hand. We conducted experiments on object states prediction and in-hand manipulation using a reinforcement learning algorithm (PPO). Results show that TacGNN is effective in predicting object-related states during manipulation by decreasing the RMSE of prediction to 0.096cm comparing to other methods, such as MLP, CNN, and GCN. Finally, the robot hand could finish an in-hand manipulation task solely relying on the robotic own perception - tactile sensing and proprioception. In addition, our methods are tested on three tasks with different difficulty levels and transferred to the real robot without further training.