(277) A Methodology Study to Evaluate External Comparator Arm Study Results Versus Randomised Controlled Trial Results: Multiple Myeloma Case Study

Publication Number: 951

Background: Single-arm trials (SATs) with an external comparator arm (ECA) have been introduced as an alternative to randomised controlled trials (RCTs) in cancer drug development.

Objectives: To compare estimated effect sizes for the endpoint overall survival (OS) utilising data from a completed RCT and real-world data (RWD) forming an ECA.

Methods: Estimated effect sizes from a completed RCT in patients with multiple myeloma (MM) were compared with an ECA. RWD for MM was obtained from a network of regional community health systems in the US. RWD were harmonised with RCT data, considering eligibility criteria, treatments compared, covariates and the outcome OS. The data from the experimental arm of the RCT was then used as if it was a SAT and analysed with the RWD ECA using propensity score methods to account for the imbalance of potential confounders between the SAT and the ECA. Multiple imputation was used to derive a complete set of baseline covariates for each patient. An effort was made to also generate a dataset based on RWD similar to the treatment arm of the RCT.

Results: The MM RCT dataset included 242 and 229 patients in the treatment and control arms, respectively. After applying prioritized eligibility (and treatment-related) criteria, there were 461 patients in the ECA dataset and 284 patients in the RWD treatment arm dataset. The Hazard Ratio (HR) from the original RCT was 0.71 (95% confidence interval [CI]=0.52-0.96). The unweighted HR comparing the SAT with the ECA was 0.63 (95% CI=0.48-0.82). Propensity score weighting resulted in improved balance and standardised mean differences were observed to be < 0.1 for the weighted analyses. The HR was 0.78 (95% CI=0.59-1.02) by average treatment effect (ATE) weighting. An HR of 0.86 and 0.88, respectively, was observed by average treatment effect on the treated (ATT, 95% CI=0.62-1.17) and average treatment effect on the overlap population (ATO, 95% CI=0.66-1.17) weighting. The ATE weighted comparison between the RWD treatment group and the ECA showed that the direction of treatment effect for comparison within the RWD was similar to that of the RCT and well within the CI of the estimated effect for the RCT (HR=0.85, 95% CI=0.67-1.07).

Conclusions: This MM case study showed that despite limitations of the RWD it was possible to replicate the results of the RCT by using a SAT with an ECA. Successful application of the key eligibility criteria of the RCT seemed to contribute significantly to the conditional exchangeability of the populations. The weighted results for OS were similar to those from the original RCT. This case study is part of a larger study that also includes a prostate cancer case study and simulations.