2021/12/27 by Deepak Alapatt, Pietro Mascagni, Alapatt, Deepak +17 · 1 citation
Medicine · #Artificial Intelligence in Healthcare and Education #Colorectal Cancer Screening and Detection #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Neural and Evolutionary Computing (cs.NE) #Surgical Simulation and Training
paper · pdf · doi:10.48550/arxiv.2112.13815
openalex publication_date 2021/12/27 · openalex created_date 2021/12/31 · openalex updated_date 2026/07/28
A major obstacle to building models for effective semantic segmentation, and particularly video semantic segmentation, is a lack of large and well annotated datasets. This bottleneck is particularly prohibitive in highly specialized and regulated fields such as medicine and surgery, where video semantic segmentation could have important applications but data and expert annotations are scarce. In these settings, temporal clues and anatomical constraints could be leveraged during training to improve performance. Here, we present Temporally Constrained Neural Networks (TCNN), a semi-supervised framework used for video semantic segmentation of surgical videos. In this work, we show that autoencoder networks can be used to efficiently provide both spatial and temporal supervisory signals to train deep learning models. We test our method on a newly introduced video dataset of laparoscopic cholecystectomy procedures, Endoscapes, and an adaptation of a public dataset of cataract surgeries, CaDIS. We demonstrate that lower-dimensional representations of predicted masks can be leveraged to provide a consistent improvement on both sparsely labeled datasets with no additional computational cost at inference time. Further, the TCNN framework is model-agnostic and can be used in conjunction with other model design choices with minimal additional complexity.