PC3112: Feasibility of Eye Tracking to Measure Expertise in Pediatric Point of Care Echocardiography: A Pilot Study

Background: Physician-performed point of care ultrasound (POCUS) is a commonly employed diagnostic tool. POCUS training programs, however, are neither standardized nor as robust as dedicated sonographer training. In addition, there is a lack of consensus national guidelines for establishing competence. There is therefore a need for objective metrics to help determine proficiency in order to guarantee patient safety. Eye tracking technology, in which a device measures reflections of light off the eyes to determine real-time visual gaze location has been employed by athletics, aviation, and the military to objectively determine levels of expertise. This pilot study explored whether eye tracking metrics could reliably predict physician accuracy in interpreting echocardiography images.

Methods: We recruited pediatric subspecialty attending physicians without training in echocardiography (novices) and pediatric subspecialty attendings with advanced training in echocardiography (experts) at a 116-bed tertiary children’s hospital. Participants were each shown ten 6-second echocardiogram clips, 5 of which had pathology and 5 of which did not, and were given a multiple choice interpretation while a screen-based eye tracker (Tobii Nano Pro; Tobii; Stockholm, Sweden) recorded their eye movements. For clips with pathology, an area of interest (AOI) was predefined by the study team. The primary outcomes assessed the correlations between accuracy of diagnosis and time to first fixation (TTFF) on the area of interest as well as first fixation duration (FFD) on the AOI. The secondary outcomes assessed the correlations between pre-defined expertise and TTFF on the AOI and FFD on the AOI.

Results: 10 participants (5 novices and 5 experts) completed this study. Mean TTFF was shorter among correctly identified pathology (mean 220.8 ms) than incorrectly identified pathology (mean 1064.0 ms) (p < 0.001). Participants with lower TTFF also had a higher likelihood of choosing the correct diagnosis (0-250 ms: 14/14, 250-325 ms: 13/15, >325 ms: 4/14) (p=0.003, post-hoc comparisons of 0-250 ms and 250-325 ms with >325 ms group, p = 0.006 and 0.009 respectively). Mean FFD was significantly shorter among correctly identified pathology (mean 303.6 ms) than incorrectly identified pathology (mean 621.4 ms) (p < 0.001). Pre-defined experts also had significantly lower TTFF (p=0.03) and FFD (p=0.05) when compared with novices.

Conclusion: Eye tracking metrics accurately predict correct responses in this cohort of physicians. The metrics used also differentiate levels of expertise in POCUS. This data also suggests a preliminary threshold of 325 ms to first fixate on an AOI on echocardiogram clips, after which the likelihood of correctly identifying pathology substantially decreases. These findings merit further study and may have applications to both educational interventions and credentialing.