(147) A Comparison of Structured Algorithms for Identifying Men with Metastatic Prostate Cancer Within the Veterans Affairs Health Care System

Publication Number: 1192

Background: Correctly classifying metastatic prostate cancer (mPC) is critical but challenging in population-based research. As chart review of electronic health records is not feasible on a population-based scale, accurate automated methods are needed to perform impactful research.

Objectives: This study compared the performance of two automated algorithms based on ICD codes and treatment records to identify mPC with chart review as the gold standard.

Methods: This national cohort study was conducted in the Veterans Affairs (VA) Health System using information from claims as well as unstructured medical notes from charts. Adult males diagnosed with PC (defined by 2 or more ICD-9:185. or ICD-10:C61. codes on separate dates) between 2012-2017 who had chart review data available were included. The study population was comprised of men with varying risk for metastases, including men treated with therapies only approved for metastatic or castration resistant PC (n=134), men with castration sensitive (CS) PC with PSA ≥20 ng/mL (n=299), and men with CSPC with PSA < 20 ng/mL (n=289); both groups of CSPC men did not receive therapy beyond ADT and/or primary therapy. Two algorithms were used to identify mPC: 1) ICD-9/10 codes for secondary malignant neoplasms (ICD-alone) and 2) ICD-9/10 codes and/or common mPC treatments (ICD + treatment). Using chart review as the gold standard, sensitivity and specificity were calculated for ICD-alone and ICD + treatment algorithms. The positive predictive value (PPV) and negative predictive value (NPV) were estimated across a range of potential mPC prevalence.

Results: Of 722 men with PC, 177 were classified as mPC from chart review compared to 174 by the ICD-alone and 203 by the ICD + treatment algorithms.
Sensitivity and specificity for the ICD-alone algorithm were 72.9 (95% confidence interval: 65.7-79.3) and 91.7 (89.1-93.9), respectively; for the ICD + treatment algorithm they were 80.2 (73.6-85.8) and 88.8 (85.9, 91.3), respectively.
Applying these values and assuming a range of mPC prevalence between 10-30%, the PPV ranged from 49.5-79.1 for ICD-alone and from 44.3-75.4 for ICD + treatment; the NPV ranged from 88.8 to 96.8 for ICD-alone and from 91.3 to 97.6 for ICD + treatment.

Conclusions: In the VA system, an algorithm based on ICD codes for secondary malignant neoplasms alone had high specificity but modest sensitivity. Adding treatments for metastatic PC to the ICD-alone algorithm enhanced sensitivity with little impact on specificity, and thus is a feasible approach to identify mPC in the absence of chart review. In the future, building upon these algorithms using natural language processing is likely to further improve diagnostic accuracy, though this requires formal validation.