(315) Machine Learning to Predict Safety Signals Using Molecular Similarity and Disproportionate Reporting Rates

Publication Number: 989

Background: Disproportionality analysis, a way to quantify unexpectedness, is the standard approach in statistical signal detection. To improve signal detection, information can be leveraged from similar drugs. By using safety profiles of chemically similar drugs, the aim is to find signals earlier as chemical similarity gives additional support resulting in less case safety reports required to detect signals. Furthermore, leveraging chemical similarity may propose a mechanism of action for the suspected Adverse Event (AE).

Objectives: Predict American Structured Product Label (SPL) changes using machine learning based on known AE of structurally similar drugs and disproportionate reporting rates.

Methods: To create a dataset of drug-AE pairs, SPLs were mined with advanced natural language processing algorithms. We excluded combination products, biologics, herbals, vaccines, as well as dermatological and ophthalmic drugs. A subset of the included drugs were test drugs which had a label change to their SPL within a defined time window. The dataset had 3528 drug-AE pairs, composed of positives (label changes) and negatives. Drugs were chemically represented as Morgan fingerprints of radius 2 and their similarity was computed using Dice coefficient. The Information Component (IC) was used as a measure of disproportionality between observed and expected reporting of drug-AE pairs. Features for the model included chemical similarity, presence of an AE in the comparator label, and IC values computed in VigiBase, the World Health Organization’s global database of reported AE of medicinal products. The machine learning model chosen was a random forest, trained and evaluated using 10-fold cross-validation.

Results: Using only positive IC025 without a machine learning model as a baseline model resulted in an f1 score of 0.41 with precision of 0.84 and recall of 0.26. Feeding IC values into the random forest model resulted in an f1 of 0.64, with precision (0.64) and recall (0.66). If only chemical similarity and comparator label information is used with the random forest model, f1 of 0.66 is found with similar precision (0.66) and recall (0.68). Both random forest models perform much better in recall than the baseline model. Best results are found when combining IC and chemical information, resulting in an f1 of 0.74 with precision of 0.72 and recall of 0.76. Other chemical representations and machine learning models performed similarly.

Conclusions: When only using IC values, recall is low. Considering drug chemical similarity or IC values in a machine learning model improves prediction of label changes. Best performances are achieved when chemical information and disproportionality are used together, showing that chemical information could support timely signal detection.