Recognizing Surgical Activities with Recurrent Neural Networks

2016/06/20 by Robert DiPietro, Colin Lea, DiPietro, Robert +13
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #cs.CV

paper · pdf · doi:10.48550/arxiv.1606.06329

Conditionally accepted at MICCAI 2016

arxiv created 2016/06/22 · arxiv updated 2016/06/23

Abstract

We apply recurrent neural networks to the task of recognizing surgical activities from robot kinematics. Prior work in this area focuses on recognizing short, low-level activities, or gestures, and has been based on variants of hidden Markov models and conditional random fields. In contrast, we work on recognizing both gestures and longer, higher-level activites, or maneuvers, and we model the mapping from kinematics to gestures/maneuvers with recurrent neural networks. To our knowledge, we are the first to apply recurrent neural networks to this task. Using a single model and a single set of hyperparameters, we match state-of-the-art performance for gesture recognition and advance state-of-the-art performance for maneuver recognition, in terms of both accuracy and edit distance. Code is available at https://github.com/rdipietro/miccai-2016-surgical-activity-rec .

Related

• Multi-Task Recurrent Neural Network for Surgical Gesture Recognition and\n Progress Prediction
• MS-TCRNet: Multi-Stage Temporal Convolutional Recurrent Networks for Action Segmentation Using Sensor-Augmented Kinematics
• Surgical Gesture Recognition Based on Bidirectional Multi-Layer Independently RNN with Explainable Spatial Feature Extraction
• Hierarchical Semi-Supervised Learning Framework for Surgical Gesture Segmentation and Recognition Based on Multi-Modality Data
• Recurrent and Spiking Modeling of Sparse Surgical Kinematics