2020/01/19 by Chandrakanth Jayachandran Preetha, Preetha, Chandrakanth Jayachandran, Jonathan Kloss +13
Computer Science · Engineering · Medicine · #68T45 #Augmented Reality Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Image and Video Processing (eess.IV) #Robotics and Sensor-Based Localization #Surgical Simulation and Training #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2001.06894
openalex publication_date 2020/01/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Minimally Invasive Surgery (MIS) techniques have gained rapid popularity\namong surgeons since they offer significant clinical benefits including reduced\nrecovery time and diminished post-operative adverse effects. However,\nconventional endoscopic systems output monocular video which compromises depth\nperception, spatial orientation and field of view. Suturing is one of the most\ncomplex tasks performed under these circumstances. Key components of this tasks\nare the interplay between needle holder and the surgical needle. Reliable 3D\nlocalization of needle and instruments in real time could be used to augment\nthe scene with additional parameters that describe their quantitative geometric\nrelation, e.g. the relation between the estimated needle plane and its rotation\ncenter and the instrument. This could contribute towards standardization and\ntraining of basic skills and operative techniques, enhance overall surgical\nperformance, and reduce the risk of complications. The paper proposes an\nAugmented Reality environment with quantitative and qualitative visual\nrepresentations to enhance laparoscopic training outcomes performed on a\nsilicone pad. This is enabled by a multi-task supervised deep neural network\nwhich performs multi-class segmentation and depth map prediction. Scarcity of\nlabels has been conquered by creating a virtual environment which resembles the\nsurgical training scenario to generate dense depth maps and segmentation maps.\nThe proposed convolutional neural network was tested on real surgical training\nscenarios and showed to be robust to occlusion of the needle. The network\nachieves a dice score of 0.67 for surgical needle segmentation, 0.81 for needle\nholder instrument segmentation and a mean absolute error of 6.5 mm for depth\nestimation.\n