(834) Natural Language Processing of Admission Notes to Predict Postpartum Hemorrhage

Artificial intelligence is being integrated into practice to support clinical decision-making. The objective of this work was to determine if natural language processing (NLP) of admission notes can predict postpartum hemorrhage (PPH) during the delivery admission, a common obstetric complication leading to severe maternal morbidity.

Study Design:

This is a retrospective study of individuals admitted for delivery between July 2016 and December 2020 at a single academic hospital. The outcome of interest was postpartum hemorrhage, defined as estimated blood loss ≥1000 mL. Admission notes were prepared for text processing by excluding stop words, punctuation, terms that occurred very frequently ( >70%) and infrequently ( < 1%). The ‘bag of words’ approach with a LASSO model was used to determine if notes were predictive of PPH. Single word vocabularies were trained in 75% of the data and model performance examined in the withheld 25% (test set). Models were then externally validated in a separate hospital within the same health system. Areas under the receiver operating characteristic curve (AUC) were calculated to assess for model discrimination, and calibration plots were generated.

Results:

The analysis included 12,590 patients in the derivation group and 20,775 patients in the external validation group. The rates of PPH were 8.2% in the test set and 8.7% in the validation set, respectively. AUCs were 0.74 (95% CI 0.71, 0.77) and 0.67 (95% CI 0.66, 0.69) in the test and validation set. The calibration plots revealed a group of patients at significantly higher risk of PPH (those in the top decile of predicted risk compared to others): rates of PPH 27.7% vs 6.2% and 23.0% vs 7.1% in the test and validation sets, respectively (p < 0.001).

Conclusion:

NLP of EHR documentation generated during routine clinical care identified individuals at highest risk for PPH. Next steps involve comparing predictions to clinician assessment and piloting the model in clinical practice. If successful, this type of tool could ultimately be incorporated to provide real-time information to care teams.