2023/07/13 by Vincent Schorp, Schorp, Vincent, Will Panitch +15
Engineering · Medicine · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Robotics (cs.RO) #Soft Robotics and Applications #Surgical Simulation and Training #Surgical Sutures and Adhesives
paper · pdf · doi:10.48550/arxiv.2307.06845
openalex publication_date 2023/07/13 · openalex created_date 2023/07/15 · openalex updated_date 2026/07/28
Accurate 3D sensing of suturing thread is a challenging problem in automated surgical suturing because of the high state-space complexity, thinness and deformability of the thread, and possibility of occlusion by the grippers and tissue. In this work we present a method for tracking surgical thread in 3D which is robust to occlusions and complex thread configurations, and apply it to autonomously perform the surgical suture "tail-shortening" task: pulling thread through tissue until a desired "tail" length remains exposed. The method utilizes a learned 2D surgical thread detection network to segment suturing thread in RGB images. It then identifies the thread path in 2D and reconstructs the thread in 3D as a NURBS spline by triangulating the detections from two stereo cameras. Once a 3D thread model is initialized, the method tracks the thread across subsequent frames. Experiments suggest the method achieves a 1.33 pixel average reprojection error on challenging single-frame 3D thread reconstructions, and an 0.84 pixel average reprojection error on two tracking sequences. On the tail-shortening task, it accomplishes a 90% success rate across 20 trials. Supplemental materials are available at https://sites.google.com/berkeley.edu/autolab-surgical-thread/ .