MP41-08: Advanced techniques to improve video-based automation of suturing skills assessment

Andrew J Hung*, Los Angeles, CA, Richard Bao, Pasadena, CA, Idris O Sunmola, Evanston, IL, De-An Huang, Santa Clara, CA, Jessica H Nguyen, Los Angeles, CA, Anima Anandkumar, Pasadena, CA

Introduction: In our previous work, we showed that technical skill scores, obtained via manual ratings, are stronger predictors of patient continence recovery following robot-assisted radical prostatectomy (RARP) than other objective measures of surgeon performance. In this study, we automate the scoring of suturing technical skills through computer vision (CV) methods, laying the groundwork to provide surgeons with skills feedback.

Methods: 22 surgeons completed a suturing exercise three times on the Mimic FlexVR™ simulator. Instrument kinematic data (XYZ coordinates of each instrument and pose) were captured at 30 Hz. Videos were segmented into four sub-stitch phases (Needle handling, Needle targeting, Needle driving, Needle withdrawal) and labeled according to four technical skill domains (Needle positioning, Needle entry, Needle driving, and Needle withdrawal) by 3 human raters after standardized training. The CV framework extracted features from RGB and optical flow frames (inputs) using a pre-trained AlexNet to generate a skills assessment classification.  To improve model performance, our framework also exploited auxiliary supervision (using instrument kinematic data during the training phase only) and an attention mechanism (Fig. 1a).

Results: This study included 15 expert surgeons (median caseload 300 [IQR 165-750]) and 7 training surgeons (0 [IQR 0-8]). In all, 226 virtual sutures were captured. Automated assessments for Needle positioning performed best with the simplest CV approach (video only; AUC 0.691; Fig.1b). Assessment of the remaining suturing skill domains benefited from each of the additional CV components (auxiliary supervision and attention) when deployed separately (0.604 > AUC < 0.794). Deploying such components in unison resulted in the best performance, particularly for Needle driving and Needle withdrawal skill domains (AUC 0.960 and 0.879, respectively).

Conclusions: This study demonstrated the best performance of automated assessment of suturing technical skills to date. Advanced techniques including auxiliary supervision and attention mechanisms, CV-based assessments can provide automated feedback to training surgeons.

Source of Funding: This study is supported in part by the National Cancer Institute under Award Number 1R01CA251579-01A1.