Introduction: Cognitive Load (CL) can affect surgeon performance and the ability of training surgeons to receive feedback. We investigate the value of CL for detecting surgical errors in robot-assisted surgery (RAS) with a machine learning (ML)-based model. Methods: We use 2 datasets (DS1, DS2) where 24 and 44 surgeons, respectively, performed a simulated dry-lab tissue dissection task on a live daVinci Xi surgical robot. The data contains video and per-second Index of Cognitive Activity (ICA), derived from pupillary change tracked via Tobii Eyetrackers. DS1 includes surgeons with low, medium, and high experience (by hours of RAS) as well as annotated tissue Tear (n=70) and Puncture (n=62) errors. DS2 contains data from low experience surgeons only and annotations for Tears (n=445) and Punctures (n=527). We extract 5 sec of video and varying history of ICA (from 5 to 60 sec) before and around human-labeled errors (positive label). We randomly sample the remaining data (negative label). We train variations of ML architecture consisting of video processing (pretrained Resnet18) and/or ICA processing (LSTM). We train the model on 3 different 80/20 train-test between-participant random splits. Results: When including surgeons of mixed experience levels (DS1), ICA alone (AUC=64.28) is competitive with video alone (AUC=62.59) in detecting surgical errors (Fig 1A). However, the use of ICA in combination with video improves AUC in both datasets with gains of 5.55 (DS1) and 3.24 (DS2). History of ICA up to 20-30 seconds seems beneficial for error detection, after which point there is no further gain (Fig 1B). This is likely attributed to gradual build-up in CL leading to an error, and low-frequency of ICA measurements (1/sec), which may hinder extracting useful information from short time windows. The challenge of extracting meaningful information seems supported by Fig 1C, where providing processed ICA is beneficial (AUC=62.95) on top of raw ICA measurements (AUC=58.88). Conclusions: A surgeon’s CL in the context of surgical errors in RAS education is important. We show how CL is particularly valuable when a dataset includes mixed-experience surgeons. We also show the importance of ICA history length and pre-processing. Further work will explore better methods of extracting valuable features from CL data and its importance in surgery. SOURCE OF Funding: This material is based upon work supported by the National Science Foundation under Grant # 2030859 to the Computing Research Association for the CIFellows Project. Additionally, research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under Award Number R01CA273031. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
