Tactile Object Pose Estimation from the First Touch with Geometric Contact Rendering

2020/12/09 by Eric Valls, Bauza, Maria, Bryan Lim +7 · 7 citations
Computer Science · Engineering · Neuroscience · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Interactive and Immersive Displays #Machine Learning (cs.LG) #Robot Manipulation and Learning #Robotics (cs.RO) #Tactile and Sensory Interactions

paper · pdf · doi:10.48550/arxiv.2012.05205

openalex publication_date 2020/12/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we present an approach to tactile pose estimation from the first touch for known objects. First, we create an object-agnostic map from real tactile observations to contact shapes. Next, for a new object with known geometry, we learn a tailored perception model completely in simulation. To do so, we simulate the contact shapes that a dense set of object poses would produce on the sensor. Then, given a new contact shape obtained from the sensor output, we match it against the pre-computed set using the object-specific embedding learned purely in simulation using contrastive learning. This results in a perception model that can localize objects from a single tactile observation. It also allows reasoning over pose distributions and including additional pose constraints coming from other perception systems or multiple contacts. We provide quantitative results for four objects. Our approach provides high accuracy pose estimations from distinctive tactile observations while regressing pose distributions to account for those contact shapes that could result from different object poses. We further extend and test our approach in multi-contact scenarios where several tactile sensors are simultaneously in contact with the object. Website: http://mcube.mit.edu/research/tactile_loc_first_touch.html

Cited by

• Residual Rotation Correction using Tactile Equivariance
• NeuralTouch: Neural Descriptors for Precise Sim-to-Real Tactile Robot Control
• Spatially anchored Tactile Awareness for Robust Dexterous Manipulation
• Representing Data in Robotic Tactile Perception -- A Review
• UniTac2Pose: A Unified Approach Learned in Simulation for Category-level Visuotactile In-hand Pose Estimation
• Object Pose Estimation through Dexterous Touch
• Tac2Pose: Tactile object pose estimation from the first touch

Related

• Tac2Pose: Tactile object pose estimation from the first touch
• Object Pose Estimation through Dexterous Touch
• Collision-aware In-hand 6D Object Pose Estimation using Multiple Vision-based Tactile Sensors
• Learning Tactile Models for Factor Graph-based Estimation
• Learning the sense of touch in simulation: a sim-to-real strategy for\n vision-based tactile sensing