MP10-15: Building a Real-Time Feedback System to Improve Performance on a Robotic Tissue Dissection Task

Jasper Laca*, Los Angeles, CA, Rafal Kocielnik, Pasadena, CA, Jessica H Nguyen, Ryan Tsang, Jonathan You, Alvin Hui, Los Angeles, CA, Anima Anandkumar, Pasadena, CA, Andrew J Hung, Los Angeles, CA

Introduction: An automated error prevention system can potentially provide surgical trainees with optimal real-time feedback during training and live surgery. In this study, we determine efficacy of real-time feedback during a robotic dissection task.

Methods: 45 medical students (no surgical experience) completed 2 dissection tasks on a live daVinci Xi Surgical Robot (Fig. 1a). Videos were prospectively reviewed by 4 manual raters to identify surgical errors (e.g., fruit puncture) and technical skills, as measured by Global Evaluative Assessment of Robotic Skills (GEARS). Following the 1st task, participants were stratified into control group (CG) or feedback group (FG). Feedback consisted of 8 pre-recorded audio phrases, manually triggered by a proctor when a corresponding error occurred. Feedback contained the following components: warning signal; identification of non-ideal behavior; proposed mitigation; longer-term impact (e.g., "Be cautious with how much force you are applying with your tools, be as gentle as possible when appropriate, otherwise you risk damaging the tissue."). Participants from each group were further labeled as under-performers (UP) or innate performers (IP), based on a median split of errors and GEARS from the 1st task. T-test and Mann-Whitney analysis were used to compare performance between groups.

Results: Raters achieved intraclass correlation coefficients of 0.84 & 0.81 for errors classification and GEARS, respectively (p < 0.001). Initial performance, based on number of errors and GEARS, were comparable between CG (n=22) and FB (n=21) (median number of errors: 19.0 vs 19.5, p=0.93; GEARS score: 13.85 vs 14.06, p=0.82). During the 2nd task, FG had lower median error counts (6.5 vs 12.5, p<0.05) and higher GEARS score (18.24 vs.16.10, p=0.01). All IP exhibited similar improvements between the 1st and 2nd task, regardless of feedback (p=0.97) (Fig. 1b). On the other hand, UP in the FG performed significantly better than their CG counterparts during the 2nd task (p < 0.01), suggesting that feedback was particularly valuable for UP.

Conclusions: Real-time feedback can reduce surgical errors and improve robotic technical skills, especially for UP. Future research should look to expand real-time feedback by identifying relevant errors for feedback across different procedures and experience groups.

Source of Funding: None