Toward Artificial Palpation: Representation Learning of Touch on Soft Bodies

2025/11/20 by Zohar Rimon, Rimon, Zohar, Elisei Shafer +7
Engineering · Neuroscience · #Advanced Sensor and Energy Harvesting Materials #FOS: Computer and information sciences #Machine Learning (cs.LG) #Robot Manipulation and Learning #Tactile and Sensory Interactions

paper · pdf · doi:10.48550/arxiv.2511.16596

openalex publication_date 2025/11/20 · openalex created_date 2025/11/23 · openalex updated_date 2026/07/28

Abstract

Palpation, the use of touch in medical examination, is almost exclusively performed by humans. We investigate a proof of concept for an artificial palpation method based on self-supervised learning. Our key idea is that an encoder-decoder framework can learn a representation from a sequence of tactile measurements that contains all the relevant information about the palpated object. We conjecture that such a representation can be used for downstream tasks such as tactile imaging and change detection. With enough training data, it should capture intricate patterns in the tactile measurements that go beyond a simple map of forces -- the current state of the art. To validate our approach, we both develop a simulation environment and collect a real-world dataset of soft objects and corresponding ground truth images obtained by magnetic resonance imaging (MRI). We collect palpation sequences using a robot equipped with a tactile sensor, and train a model that predicts sensory readings at different positions on the object. We investigate the representation learned in this process, and demonstrate its use in imaging and change detection.

Citations

• Breast Lump Detection and Localization with a Tactile Glove Using Deep Learning
• AnyTouch: Learning Unified Static-Dynamic Representation across Multiple Visuo-tactile Sensors
• Sparsh: Self-supervised touch representations for vision-based tactile sensing
• Deep Learning for Accelerated and Robust MRI Reconstruction: a Review
• Using Fiber Optic Bundles to Miniaturize Vision-Based Tactile Sensors
• General In-Hand Object Rotation with Vision and Touch
• Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Play
• FingerSLAM: Closed-loop Unknown Object Localization and Reconstruction from Visuo-tactile Feedback
• Deep Movement Primitives: toward Breast Cancer Examination Robot
• Masked Autoencoders Are Scalable Vision Learners
• Bootstrap your own latent: A new approach to self-supervised Learning
• U-Net: Convolutional Networks for Biomedical Image Segmentation
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Generalized autocalibrating partially parallel acquisitions (GRAPPA)
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• A Threshold Selection Method from Gray-Level Histograms

Related

• FELT: Generating Tactile Signals from Vision for Visuo-Tactile Manipulation
• Learning the sense of touch in simulation: a sim-to-real strategy for\n vision-based tactile sensing
• Unsupervised preprocessing for Tactile Data
• Deep Learning for Tactile Understanding From Visual and Haptic Data
• Multimodal Sensing for Robot-Assisted Sub-Tissue Feature Detection in Physiotherapy Palpation