(257) Totality of Evidence Evaluating the Effectiveness of Repurposed Therapies for COVID19: What Can We Learn from Real-World Studies & Randomized Controlled Trials?

Publication Number: 931

Background: The world remains vulnerable to future pandemics of infectious diseases. Rapid and robust strategies to evaluate the efficacy of existing and new pharmacotherapeutic interventions will thus be required. Observational ‘Real-World Studies’ (RWS) report quickly and would have value were they to yield results of comparable reliability to those from randomised controlled trials (RCTs) which take longer to complete.

Objectives: The COVID-19 situation offers a unique opportunity to determine the contribution of RWS relative to RCTs and how to optimise research efforts in future pandemics.

Methods: We analysed summary level data from RCTs and RWS from publicly available sources reported before 24 February 2022 and housed in the CODEx COVID-19 database. Included were studies comparing treatment + unrestricted standard of care (SoC) vs SoC, with a total of >500 patients across >5 independent unique studies, and which reported all-cause mortality. Analyses used random effects models to pool odds ratios and multivariate mixed model meta-regression in order to identify factors explaining heterogeneity. Mixed effects meta-analysis was deployed to characterise the time course of the cumulative odds ratio for mortality for RCTs alone, RWS alone, and RWS and RCTs combined. All steps were quality controlled.

Results: We analysed 249 RWS and RCTs across eight treatment regimens for COVID 19. RWS yielded more heterogeneous results, and generally overestimated the effect size subsequently seen in RCTs - a finding only partially explained by a few study level factors: RWS that were imbalanced for age, gender and disease severity, as well as RWS only reporting mortality at 2 weeks or less. Smaller studies of either type contributed negligibly. The reporting time course of the studies indicate that larger RCTs drive conclusive decisions regarding clinical benefit, with limited value from RWS.

Conclusions: When evaluating therapies in future pandemics, we recommend: 1) large RCTs, especially the early deployment of platform studies; 2) any RWS should be large, with high quality design and analysis; 3) reporting standards and data standards for primary endpoints, explanatory factors and key subgroups should be improved; 4) appropriate incentives to enable access to patient-level data and; 5) an overall aggregate view of all available results should be available at any given time.