(E) Changes in Sensitivity and Positive Predictive Value in Phenotype Algorithms for Myocardial Infarction and Ischemic Stroke Over Time in Multiple Observational Databases

Publication Number: 663

Background: Temporal trends in the performance characteristics, i.e., sensitivity, specificity, and positive (PPV) and negative predictive value have not been observed for phenotype algorithms for myocardial infarction (MI) or ischemic stroke.

Objectives: The objective of this study was to examine changes in sensitivity and PPV in phenotype algorithms for MI and stroke over time in multiple observational databases.

Methods: We analyzed data from 2010 to 2019 from 4 US administrative claims databases (IBM® MarketScan® Commercial Claims and Encounters (CCAE), Medicare Supplemental (MDCR), and Multi-State Medicaid (MDCD); Optum® Clinformatics® de-identified (Optum)), each transformed to the Observational Medical Outcomes Partnership Common Data Model. MI and ischemic stroke algorithms were defined based on inpatient diagnosis codes. We used PheValuator to estimate sensitivity (Sn) and PPV by developing a probabilistic reference standard based on a large-scale predictive model. Predictive models were informed by diagnostic conditions, drug exposures, laboratory measurements, and clinical observations reported in the 30 days following the index date (Cases: MI/Stroke inpatient visit; Non-cases: randomly selected inpatient visit) in subjects 18 years and older. Performance was assessed overall and within each year.

Results: For CCAE, from 2010-19, the Sn for MI (N=42,627) was 67% and PPV was 84%; for ischemic stroke (N=26,760), Sn was 56%, PPV was 96%. For Sn, we found an increasing trend for both MI (2010: Sn 53%, N=19,437; 2019: 68%, N=15,715) and ischemic stroke (2010: sensitivity 48%, N=12,862; 2019: 66%, N=10,858). For PPV, we found a decreasing trend for both MI (2010: PPV 93%, 2019: 86%) and ischemic stroke (2010: PPV 97%, 2019: 94%). The observed patterns were consistently observed in MDCD, MDCR, and Optum. For example, for MI Sn: MDCD 54 (2010) to 76% (2019); MDCR 59 to 75%; Optum 68 to 80%; for PPV: MDCD 94 to 75%; MDCR 92 to 76%; Optum 94 to 78%.

Conclusions: Using PheValuator, we found changes in both sensitivity (increasing trend) and PPV (decreasing trend) for algorithms for MI and ischemic stroke during a 10-year period. Caution is needed if one seeks to generalize performance characteristics from validation studies previously conducted to future data.