2024/08/15 by Ni Ou, Zhuo Chen, Ou, Ni +3 · 4 citations
Engineering · Neuroscience · #Advanced Sensor and Energy Harvesting Materials #EEG and Brain-Computer Interfaces #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Robotics (cs.RO) #Tactile and Sensory Interactions
paper · pdf · doi:10.48550/arxiv.2408.08276
openalex publication_date 2024/08/15 · openalex created_date 2025/01/03 · openalex updated_date 2026/07/28
Optical tactile sensors play a pivotal role in robot perception and manipulation tasks. The membrane of these sensors can be painted with markers or remain markerless, enabling them to function in either marker or markerless mode. However, this uni-modal selection means the sensor is only suitable for either manipulation or perception tasks. While markers are vital for manipulation, they can also obstruct the camera, thereby impeding perception. The dilemma of selecting between marker and markerless modes presents a significant obstacle. To address this issue, we propose a novel mode-switchable optical tactile sensing approach that facilitates transitions between the two modes. The marker-to-markerless transition is achieved through a generative model, whereas its inverse transition is realized using a sparsely supervised regressive model. Our approach allows a single-mode optical sensor to operate effectively in both marker and markerless modes without the need for additional hardware, making it well-suited for both perception and manipulation tasks. Extensive experiments validate the effectiveness of our method. For perception tasks, our approach decreases the number of categories that include misclassified samples by 2 and improves contact area segmentation IoU by 3.53%. For manipulation tasks, our method attains a high success rate of 92.59% in slip detection. Code, dataset and demo videos are available at the project website: https://gitouni.github.io/Marker-Markerless-Transition/