(234) Challenges of Using Real-World Evidence for Regulatory Decisions: 4 Key Issues

Publication Number: 908

Background: Proposed US legislation and other actions provide for increased use of Real-World Evidence (RWE) to replace or complement clinical trials to support regulatory decisions about both safety and effectiveness of therapeutic drugs.

Objectives: Identify key issues in Real-World-Data (RWD) where potential limitations may render RWE studies insufficient to support regulatory decisions by the US Food and Drug Administration.

Methods: We reviewed selected peer-reviewed literature and regulatory documents to focus on 4 key features of RWE studies conducted in insurance billing/claims data and electronic health records to provide clinical evidence of risks or benefits: (1) Representative patient population; (2) Verified exposure/adherence to study drug; (3) Accounting for all study participants; (4) Outcomes accurately captured in diagnostic or billing codes.

Results: Representative patient populations are a challenge in the fragmented US healthcare system because major RWD sources are segmented by age (excluding most patients > 65 years in commercial claims), gender (Veterans Administration health system), or socio-economic status (Medicaid, private insurance). Exposure/adherence is uncertain because RWE studies may assume that prescribed medicines were taken continuously by patients. One verification study reported that only a fraction of new prescriptions were then filled by the patients, ranging from 45% for pain medications to 76% for antimicrobials. Accounting for participants may not detect patient deaths because most insurance RWE sources cannot distinguish whether censored patients died or left the system in good health unless the patients died in-hospital or were separately linked to a death index. Many safety and efficacy outcomes cannot be captured by diagnostic/billing codes characterizing an office or emergency department visit or hospitalization. A 2020 FDA study of a widely used asthma drug with a boxed warning about neuropsychiatric side effects found no increased risk of depression in an asthma population more than 1 million patients. The study failed to capture an adverse effect of the asthma drug well documented elsewhere because the available diagnosis codes were inadequate.

Conclusions: While RWE studies contribute valuable perspectives on many issues in drug therapy, using RWE to support regulatory decisions that can affect the health and safety of thousands require special vigilance, appropriate design, and understanding that some outcomes cannot be reliably evaluated in these data. Valid use of RWE could be expanded through upgrading the major data sources to support this important public health objective.