(141) Scalable Feature Engineering from Electronic Free Text Notes to Supplement Confounding Adjustment of Claims-Based Pharmacoepidemiologic Studies

Sunday, August 28, 2022

8:00 AM – 1:45 PM CEST

Location: Hall C

Publication Number: 1186

Presenting Author(s)

Richard Wyss, PhD
– Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States

Co-Author(s)

Joseph M. Plasek
– Brigham and Women's Hospital and Harvard Medical School, United States
Li Zhou
– Brigham and Women's Hospital and Harvard Medical School, United States
Lily G. Bessette
– Brigham and Women's Hospital, Harvard Medical School, Pittsburgh, Pennsylvania, United States
Sebastian Schneeweiss, ScD, MD, FISPE
– Professor of Medicine, BWH Div of Pharmacoepidemiology and Pharmacoeconomics, Boston, Massachusetts, United States
Jeremy A. Rassen, DSc, FISPE
– Aetion, Inc, Sharon, Connecticut, United States
Theodore Tsacogianis
– Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, United States
Joshua Lin, MD
– Boston, Massachusetts, United States

Background: Natural language processing (NLP) tools turn free-text notes (FTN) from electronic health records (EHR) into data features that can supplement confounding adjustment in pharmacoepidemiologic studies. However, current applications are difficult to scale. We used unsupervised NLP to generate high-dimensional feature spaces from FTN to improve prediction of drug exposure and outcomes compared to claims-based analyses.

Objectives: To investigate if unsupervised NLP can generate high-dimensional feature spaces from FTN to improve prediction of drug exposure and outcomes compared to claims-based analyses.

Methods: We linked Medicare claims with EHR data to generate 3 cohort studies comparing different classes of medications on the risk of cardiovascular (Study 1), renal (Study 2), and gastrointestinal (Study 3) outcomes. We used ‘bag-of-words’ to generate features for the top 20,000 most prevalent terms from FTN . We compared machine learning (ML) prediction algorithms using different sets of candidate predictors: Set 1 (39 researcher-specified variables), Set 2 (Set 1 + ML-selected claims codes), Set 3 (Set 1+ ML-selected NLP-generated features), vs. Set 4 (Set 1 + 2 + 3).

Results: The study size/treatment prevalence for Studies 1-3 was 3,529/35%, 9,571/63%, and 20,862/34%, while outcome incidence was 1.9%, 1.7%, and 1.1%, respectively. When modeling treatment choice, we observed a consistent trend across the examples: ML models utilizing Set 4 performed best followed by Set 2, Set 3, then Set 1. When modeling outcomes, there was no improvement beyond models based on Set 1 in the smaller studies (1-2) and only small improvement when adding ML-selected claims codes or FTN features to Set 1 in the larger Study 3.

Conclusions: Supplementing claims data with NLP-generated features from free text notes improved prediction of prescribing choices but had little or no improvement on clinical risk prediction. These finding have implications for strategies to improve confounding using EHR data in pharmacoepidemiologic studies.