2021/07/13 by Mitchell Doughty, Doughty, Mitchell, Karan P. Singh +3
Medicine · Computer Science · Engineering · #Surgical Simulation and Training #Augmented Reality Applications #Anatomy and Medical Technology
paper · pdf · doi:10.48550/arxiv.2107.06397
We present SurgeonAssist-Net: a lightweight framework making\naction-and-workflow-driven virtual assistance, for a set of predefined surgical\ntasks, accessible to commercially available optical see-through head-mounted\ndisplays (OST-HMDs). On a widely used benchmark dataset for laparoscopic\nsurgical workflow, our implementation competes with state-of-the-art approaches\nin prediction accuracy for automated task recognition, and yet requires 7.4x\nfewer parameters, 10.2x fewer floating point operations per second (FLOPS), is\n7.0x faster for inference on a CPU, and is capable of near real-time\nperformance on the Microsoft HoloLens 2 OST-HMD. To achieve this, we make use\nof an efficient convolutional neural network (CNN) backbone to extract\ndiscriminative features from image data, and a low-parameter recurrent neural\nnetwork (RNN) architecture to learn long-term temporal dependencies. To\ndemonstrate the feasibility of our approach for inference on the HoloLens 2 we\ncreated a sample dataset that included video of several surgical tasks recorded\nfrom a user-centric point-of-view. After training, we deployed our model and\ncataloged its performance in an online simulated surgical scenario for the\nprediction of the current surgical task. The utility of our approach is\nexplored in the discussion of several relevant clinical use-cases. Our code is\npublicly available at https://github.com/doughtmw/surgeon-assist-net.\n