(134) Methods for Drug Safety Signal Detection Using Routinely Collected Electronic Healthcare Data: A Systematic Review

Publication Number: 465

Background: Signal detection is a crucial step in the discovery of post-marketing adverse drug reactions. However, there has been a lack of studies summarizing literature on drug safety signal detection using real world data (RWD) in a comprehensive manner.

Objectives: This work aims to systematically review the methods for drug safety signal detection using RWD and their performance.

Methods: We conducted a systematic review following the PRISMA guidelines. The protocol was registered in PROSPERO. MEDLINE, EMBASE, PubMed, Web of Science, Scopus and the Cochrane Library were searched with no time restrictions until 13th July 2021. Papers were searched using 42 predefined keywords and 7 medical subject headings based on the presence of 1) routinely collected healthcare data 2) a pharmacoepidemiologic drug safety focus and 3) a signal detection related keyword, limited to original research and reviews.

Results: Of 1765 reviewed studies, 101 studies and 15 reviews were eligible for inclusion. Methods included adaptations of disproportionality measures normally associated with spontaneous reports, traditional epidemiological designs, methods specific to signal detection with RWD, such as temporal pattern discovery, and implementations of machine learning. Performance assessment was often reported in terms of area under the receiver operating curve (AUC) with a range of positive and negative controls. 7 studies presented AUC values for more than one method and a range of drug-outcome pairs, in which a total of 13 methods were used. No study compared all 13 methods. The reported AUCs ranged from 0.47 to 0.81. No method consistently outperformed all other methods. 9 papers reported AUCs for subsets of drugs and outcomes. They reported both qualitative and quantitative findings at drug or outcome level. Higher AUCs were achieved for individual outcomes compared to the average measures, up to 0.94 for the self-controlled case series with acute renal failure. Different databases, reference sets, analytic configurations, metrics and definitions for the exposures and outcomes were used for performance assessment.

Conclusions: A variety of methods have been described in the literature for signal detection with RWD. Although no method showed superior performance in each study it was evaluated in, and across all drugs and diseases, there is some suggestion that self-controlled designs can achieve better results, particularly with acute outcomes. However, limited inference can be drawn on comparative performance of methods across studies given the heterogeneity of methods evaluations. Limited transparency of reporting inhibited re-analysis to understand for which subgroups of drugs and outcomes a method was likely to perform best.