(142) Validation of Safety Outcomes in Routinely Collected Data: Lessons Learned from a Multinational Post-Approval Safety Study

Sunday, August 28, 2022

8:00 AM – 1:45 PM CEST

Location: Hall C

Publication Number: 1187

Presenting Author(s)

Patrick J. Arena
– University of California, Los Angeles, Los Angeles, California, United States

Co-Author(s)

Kui Huang, PhD
– Pfizer Inc, New York, New York, United States
Lukas Löfling
– Section for Epidemiology and Prevention, Department of Research, Cancer Registry of Norway, Norway
Shahram Bahmanyar
– Centre for Pharmacoepidemiology, Karolinska Institutet, Karolinska University Hospital SE, Sweden
Jingping Mo
– Pfizer, New York, NY, United States
Stephen E. Schachterle, PhD
– Pfizer Inc., Brooklyn, New York, United States
Anthony Nunes
– Assistant Professor, Optum, Inc., Worcester, MA, United States
Elisabeth Smits
– PHARMO Institute for Drug Outcomes Research, the Netherlands, Utrecht, Utrecht, Netherlands
Rosa Juuti
– EPID Research, Finland, Finland
Fabian Hoti
– EPID Research, Finland, Finland
Pasi Korhonen
– EPID Research, Finland, Finland
Kasper Adelborg
– Associate Professor, Department of Clinical Epidemiology, Department of Clinical Medicine, Aarhus University & Aarhus University Hospital, Aarhus, Denmark, Aarhus N, Midtjylland, Denmark
Jens Sundbøll
– Department of Clinical Epidemiology, Department of Clinical Medicine, Aarhus University & Aarhus University Hospital, Denmark
Torben Rasmussen
– Department of Respiratory Diseases and Allergology, Aarhus University Hospital, Denmark
Anders Løkke
– Department of Medicine, Little Belt Hospital, Denmark
Vera Ehrenstein, MPH, DSc
– Aarhus University, Aarhus, Denmark

Background: Due to their high internal validity, clinical trials (CTs) are the gold standard for assessing the efficacy and safety of new therapies; however, inherent limitations of CTs make it difficult to detect rare or long-term outcomes. Increasingly, safety assessments of such outcomes are based on routinely collected health data; as a result, there is a need to obtain reliable evidence concerning the validity of these data sources.

Objectives: To discuss the challenges encountered and lessons learned in conducting a validation study of safety outcomes from a post-approval investigation using routinely collected data in health databases from five countries: Denmark, Finland, the Netherlands, Sweden, and the United States of America.

Methods: We requested medical records from all primary lung cancer patients initiating crizotinib and a randomly selected subset of primary lung cancer patients initiating ceritinib, erlotinib, or gefitinib treatment. Independent CT-based case definitions for each safety outcome were used to inform the data abstracted from these routine medical records. The abstracted data for each safety outcome were reviewed by two specialist adjudicators. We then calculated the positive predictive value (PPV), negative predictive value (NPV), sensitivity, and specificity of the algorithms used to identify the safety outcomes.

Results: We requested data for 749 patients but obtained completed data abstraction results for 540 patients. There was a wide range in PPVs (range: 0.00-1.00), NPVs (0.17-1.00), and sensitivity estimates (0.00 -1.00), both within each outcome by country and between the different outcomes overall; however, ranges for specificity estimates were narrower (0.81-1.00).

Conclusions: The main lesson learned was the challenge of adapting CT-based case definitions in studies that obtain information from routine medical records: CT-based case definitions assume close patient follow-up, including the identification of clinical event manifestations and sub-clinical signs. Moreover, future researchers must consider medical chart access issues, differences between database mechanisms that affect the types of recorded information, and the nature of the conditions of interest (as rare events may be more difficult to validate).