2023/07/07 by Jessica Yin, Yin, Jessica, Paarth Shah +13 · 1 citation
Engineering · Neuroscience · #Advanced Sensor and Energy Harvesting Materials #FOS: Computer and information sciences #Robotics (cs.RO) #Soft Robotics and Applications #Tactile and Sensory Interactions
paper · pdf · doi:10.48550/arxiv.2307.03839
openalex publication_date 2023/07/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Visuotactile sensors are a popular tactile sensing strategy due to high-fidelity estimates of local object geometry. However, existing algorithms for processing raw sensor inputs to useful intermediate signals such as contact patches struggle in high-deformation regimes. This is due to physical constraints imposed by sensor hardware and small-deformation assumptions used by mechanics-based models. In this work, we propose a fusion algorithm for proximity and visuotactile point clouds for contact patch segmentation, entirely independent from membrane mechanics. This algorithm exploits the synchronous, high spatial resolution proximity and visuotactile modalities enabled by an extremely deformable, selectively transmissive soft membrane, which uses visible light for visuotactile sensing and infrared light for proximity depth. We evaluate our contact patch algorithm in low (10%), medium (60%), and high (100%+) strain states. We compare our method against three baselines: proximity-only, tactile-only, and a first principles mechanics model. Our approach outperforms all baselines with an average RMSE under 2.8 mm of the contact patch geometry across all strain ranges. We demonstrate our contact patch algorithm in four applications: varied stiffness membranes, torque and shear-induced wrinkling, closed loop control, and pose estimation.