Assistant Professor of Clinical Anesthesiology and Critical Care Medicine Children's Hospital of Philadelphia Philadelphia, Pennsylvania, United States
Abstract:
Introduction: The Intensive Care Warning Index (I-WIN) is a near-real-time clinical decision support algorithm which assesses risk of deterioration in infants with single ventricle heart disease. The index was developed from 1,028 regularly gathered EMR patient variables and analyzed with machine learning extreme-gradient boosting modeling to determine risk of clinical deterioration with up to 8 hours of advance warning. While many clinical decision support tools exist in the Cardiac ICU environment, there is a paucity of data supporting clinical impact. We sought to better understand how near-real time clinical decision support algorithms impact clinician assessment in a simulated clinical scenario.
Methods: In this simulation study using real captured CICU cases, clinicians regularly involved in CICU care reviewed 12 simulated patient cases (pulled from a bank of 24 infants admitted to the CICU with single ventricle heart disease who had clinical deterioration event, matched with 24 controls who did not have deterioration events). In each case, the simulation was randomized to containing the near-real time risk-algorithm I-WIN or treatment as usual with EMR only data. All cases vignettes contained 12-hour snapshot of regularly used EMR patient time-series vital sign trends, respiratory support, select laboratory values, and medication infusions. Cases randomly contained a display of the I-WIN risk assessment (intervention) or treatment-as-usual (EMR only) without decision support. Using provided data, clinician test users assessed patient stability and risk of deterioration for each case.
Results: A total of 13 clinical test users (n=5 fellows, n=6 nurse practitioners, n=1 physician assistant, n=1 hospitalist), completed the study. Cases with I-WIN clinical decision support (intervention) had a non-significant trend toward more accurate identification of patients with a future deterioration event, compared to treatment as usual (0.81 (SD 0.18), vs 0.68 (SD 0.17), p = 0.1). There was a non-significant trend toward improved sensitivity (0.74 vs 0.56, p 0.15) in the intervention vs EMR only cases. Clinician processing time per patient case was not significantly different in intervention vs treatment as usual groups (median 67 sec (56s) vs 65 sec (26s), p = 0.44).
Conclusion: In a small cohort of test users not previously familiar or experienced with the I-WIN platform, there were non-significant trends toward improved accuracy of diagnosis and test sensitivity. In the busy, distracted, tired, or learning clinician who may not quickly recognize data trends suggesting deteriorating physiology, clinical decision support tools may improve accuracy and timeliness in assessing clinical stability. Based on this preliminary work, further real-world studies should evaluate the incremental improvement in timely and accurate identification of deteriorating physiology when using a near-real time risk algorithm in a clinical decision support framework.
