MP41-04: Motion Metrics and Machine Learning-based Feedback in Wet-lab Laparoscopic Surgical Training

Takashige Abe*, Koki Ebina, Kiyohiko Hotta, Madoka Higuchi, Jun Furumido, Naoya Iwahara, Masafumi Kon, Kou Miyaji, Sayaka Shibuya, Yan Lingbo, Suhnsuke Komizunai, Yo Kurashima, Hiroshi Kikuchi, Ryuji Matsumoto, Takahiro Osawa, Sachiyo Murai, Sapporo, Japan, Teppei Tsujita, Yokosuka, Japan, Kazuya Sase, Tagajo, Japan, Xiaoshuai Chen, Hirosaki, Japan, Atsushi Konno, Nobuo Shinohara, Sapporo, Japan

Introduction: We believe that the enrichment of self-training and peer learning along with effective feedback are crucial components for a sustainable training program.  Our aim was to build a skill-assessment system, providing objective feedback based on the motion-metrics of laparoscopic surgical instruments.

Methods: Subjects performed tissue dissection around the aorta (Task 1), and renal parenchymal closure (Task 2), using swine organs.  Motion of surgical devices were recorded by a motion capture (Mocap) system that consists of 6 infrared cameras, and simultaneously tracked the movements of multiple surgical instruments (Figure 1a) .  Two experts scored the recorded movies, according to the formula of Global Operative Assessment of Laparoscopic Skills (GOALS).  The mean scores were utilized as objective variables in the subsequent regression analyses.  The correlations between mean GOALS scores and Mocap metrics were evaluated, and potential Mocap-metrics with a Spearman’s rank correlation coefficient value above 0.4 (e.g., path length, velocity, acceleration, and jerk of each instrument) were selected for each GOALS item estimation.  In Task 2, the item of “tissue handling” was omitted due to difficulty of evaluation.  Four regression algorithms: support vector regression (SVR), principal component analysis (PCA)-SVR, ridge regression, and partial least squares regression, were employed for automatic GOALS assessment.  Model validation was conducted by nested and repeated k-fold cross-validation, and the mean absolute error (MAE) was calculated to evaluate the accuracies of each model.

Results: A total of 45 urologic surgeons, 9 gastroenterological surgeons, 3 gynecologic surgeons, 4 junior residents, and 9 medical students voluntarily participated in the training.  Among the 4 algorithms, SVR had the highest accuracy in Task 1 (MAEmedian=2.2352, GOALS Score range: 5-25), and PCA-SVR (MAEmedian=1.2714, GOALS Score range: 4-20) in Task 2 in 100 iterations of the validation process.  Figures 1b and 1c show scatter plots of actual vs. predicted GOALS scores.

Conclusions: We developed a machine learning-based skill assessment system providing GOALS scores based on Mocap metrics with an error of approximately 1-2 points for the total score.

Source of Funding: KAKENHI