2022/09/19 by Jingxi Xu, Han Lin, Xu, Jingxi +5 · 4 citations
Engineering · Neuroscience · #Advanced Sensor and Energy Harvesting Materials #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #EEG and Brain-Computer Interfaces #FOS: Computer and information sciences #Machine Learning (cs.LG) #Robotics (cs.RO) #Tactile and Sensory Interactions
paper · pdf · doi:10.48550/arxiv.2209.08772
openalex publication_date 2022/09/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Tactile recognition of 3D objects remains a challenging task. Compared to 2D shapes, the complex geometry of 3D surfaces requires richer tactile signals, more dexterous actions, and more advanced encoding techniques. In this work, we propose TANDEM3D, a method that applies a co-training framework for exploration and decision making to 3D object recognition with tactile signals. Starting with our previous work, which introduced a co-training paradigm for 2D recognition problems, we introduce a number of advances that enable us to scale up to 3D. TANDEM3D is based on a novel encoder that builds 3D object representation from contact positions and normals using PointNet++. Furthermore, by enabling 6DOF movement, TANDEM3D explores and collects discriminative touch information with high efficiency. Our method is trained entirely in simulation and validated with real-world experiments. Compared to state-of-the-art baselines, TANDEM3D achieves higher accuracy and a lower number of actions in recognizing 3D objects and is also shown to be more robust to different types and amounts of sensor noise. Video is available at https://jxu.ai/tandem3d.