0550: Improving the Accuracy of Automated Gout Flare Ascertainment Using Natural Language Processing of Medical Records and Linked Medicare Claims Data for Real-world Comparative Effectiveness Research

Kazuki Yoshida, Tianrun Cai, Lily G Bessette, Erin Kim, Su Been Lee, Luke E Zabotka, Alec Sun, Julianna M. Mastrorilli, Theresa A. Oduol, Jun Liu, Daniel Solomon, Katherine Liao and Seoyoung Kim, Brigham and Women's Hospital, Boston, MA

Background/Purpose: Gout flares are challenging to study using large real-world data because flares are episodic with varying severity and clinical manifestations. Furthermore, there is no specific diagnosis code or unified treatment protocol for gout flares. We aimed to determine whether integrating concepts extracted from the narrative notes from the electronic health record (EHR) data using natural language processing (NLP) could improve the identification of gout flares for the purpose of comparative effectiveness research.

Methods: Using Medicare claims data (Parts A/B/D 2007-16) linked with EHR from two tertiary care centers, we selected individuals with age≥65 and ≥1 gout ICD-9 or 10 code who initiated the urate-lowering therapy (ULT). Patients were required to have ≥12 months with continuous enrollment (i.e., baseline period) before ULT initiation. The baseline period and on-treatment follow-up were segmented into 1-month units. We retrieved EHR notes for months with gout ICD codes in claims, and processed notes for NLP concepts related to gout using NLP. We selected a random sample of patients and reviewed each of their notes for the presence of acute gout flare based on the physician documentation of gout flare as the gold standard. Months containing 1+ notes with acute gout flares were considered months with events. We then used 60% of patients to train predictive models with LASSO using claims variables only, concepts only, and both combined. We evaluated the models by comparing the area under the curve (AUC) of ROCs in the 40% validation data and examined positive predictive value (PPV) at specificity thresholds of 85, 90, and 95%.

Results: We extracted and labeled 3,482 notes from 500 patients for the presence of acute gout flares, representing 839 months of follow-up time, and 280 months with gout flares. We used 93 claims variables and 84 EHR NLP concepts in LASSO. The claims-only model selected 20 variables (Table 1; e.g., steroids, colchicine, diabetes) with AUC 0.69 [95%CI 0.64,0.75] (Figure 1). The NLP concept-only model selected 17 (e.g., "gout", "pain", "arthralgia") with AUC 0.69 [95%CI 0.63,0.75]. The combined model selected 32 claims variables (e.g., steroids, NSAIDs, colchicine) and 13 NLP concepts (e.g., "gout", "pain", "arthralgia") with AUC 0.73 [95%CI 0.68,0.79]. The claims-only algorithm had a positive predictive value (PPV) of 0.64 and negative predictive value (NPV) of 0.71 at the specificity of 90%, whereas the combined algorithm had a PPV of 0.76 and NPV of 0.71 at the specificity of 95% (Table 2).

Conclusion: Although either claims variables alone or EHR NLP concepts alone achieved similar discrimination, the model with both claims and EHR variables resulted in a small improvement in the AUC. Compared to an existing rule-based claims algorithm (PPV 0.59-0.68; MacFarlane 2016 Pharmacoepi Drug Saf), our data-driven claims-only algorithm was comparable and our data-driven claims + NLP concept algorithm outperformed (PPV 0.76 at specificity 95%).
Table 1. Variables selected by LASSO.

Figure 1. Validation area under the curve (AUC) for each model.

Table 2. Performance of the claims-only and claims + NLP concept models at different specificity thresholds (85, 90, 95, 97%).
Disclosures: K. Yoshida, None; T. Cai, None; L. Bessette, None; E. Kim, None; S. Lee, None; L. Zabotka, None; A. Sun, None; J. Mastrorilli, None; T. Oduol, None; J. Liu, None; D. Solomon, None; K. Liao, None; S. Kim, Pfizer, AbbVie/Abbott, Roche, Bristol-Myers Squibb(BMS).