2021/03/16 by Megha Kalia, Kalia, Megha, Tajwar Abrar Aleef +5
Computer Science · Engineering · Medicine · #Anatomy and Medical Technology #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Image and Video Processing (eess.IV) #Machine Learning (cs.LG) #Medical Imaging and Analysis #Robotics (cs.RO) #Surgical Simulation and Training #cs.CV #cs.LG #cs.RO #eess.IV #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2103.09276
11 pages, 3 figures
arxiv created 2021/03/16 · openalex publication_date 2021/03/16 · arxiv updated 2021/03/18 · openalex created_date 2022/10/01 · openalex updated_date 2026/07/28
Surgical instrument segmentation for robot-assisted surgery is needed for accurate instrument tracking and augmented reality overlays. Therefore, the topic has been the subject of a number of recent papers in the CAI community. Deep learning-based methods have shown state-of-the-art performance for surgical instrument segmentation, but their results depend on labelled data. However, labelled surgical data is of limited availability and is a bottleneck in surgical translation of these methods. In this paper, we demonstrate the limited generalizability of these methods on different datasets, including human robot-assisted surgeries. We then propose a novel joint generation and segmentation strategy to learn a segmentation model with better generalization capability to domains that have no labelled data. The method leverages the availability of labelled data in a different domain. The generator does the domain translation from the labelled domain to the unlabelled domain and simultaneously, the segmentation model learns using the generated data while regularizing the generative model. We compared our method with state-of-the-art methods and showed its generalizability on publicly available datasets and on our own recorded video frames from robot-assisted prostatectomies. Our method shows consistently high mean Dice scores on both labelled and unlabelled domains when data is available only for one of the domains. *M. Kalia and T. Aleef contributed equally to the manuscript