(129) Validity of Adverse Clinical Outcomes in Routine Health Data for COVID-19 Vaccines and Therapies

Publication Number: 460

Background: Routine health data has been a valuable resource to generate evidence on real-world safety and effectiveness for Covid-19 vaccines and treatments. However, the robustness of the research is dependent on the validity of diagnoses used to identify the key clinical endpoints.

Objectives: To understand and evaluate the accuracy of diagnostic algorithms in routine health data relevant for Covid-19 research.

Methods: We conducted a literature search for validation studies published in English between 2015 to 2020 using North American or Western European routine health data which assessed cardiovascular, thromboembolic, respiratory and systematic inflammatory events of interest to both COVID-19 vaccines and therapeutics as listed by the Safety Platform for Emergency Vaccines (SPEAC). We included studies with a Quality Assessment of Diagnostic Accuracy Studies (QUADAS) score >7 that reported positive predictive values (PPV). We extracted the study population, data source, codes, algorithm, reference standard, PPV and calculated median and interquartile ranges for the PPVs found.

Results: We identified 43 studies. The events most commonly validated were myocardial infarction (MI), thromboembolic events and stroke. The median PPV for identifying MI was 80% (IQR 22%) (n=13 studies). The PPVs for venous thromboembolism (VTE) were more variable (range 31%-92; median 73%; n=11 studies). Within VTE, the median PPV for pulmonary embolism was higher than for deep-vein thrombosis. We observed PPVs >70% for atrial fibrillation (AF) (n=5). The PPVs for general stroke were variable, but highest for ischaemic stroke (PPVs for ischaemic stroke ≥85%; n=2 studies). Several events of interest (arrythmias, pneumonitis, myocarditis, respiratory distress syndrome) had only a single validation study available; while others events of interest critical in the early pandemic period (cytokine release syndrome and multi organ dysfunction syndrome) had none. Data on renal failure were difficult to interpret due to lack of uniform definitions across studies.

Conclusions: Accurate validated algorithms were available for MI, VTE, ischaemic stroke and AF, while there was considerable variability in PPV for general stroke depending on the data source and healthcare setting. There remains a lack of validation studies for many cardiovascular, respiratory and systemic adverse clinical events that emerged of interest during the early part of the pandemic. More published validation studies and algorithms are needed to improve the quality of real-world safety and effectiveness studies for Covid-19 vaccines and treatments.