(159) Development of a Feasibility and Conceptual Design Framework for Comparative Effectiveness Research

Publication Number: 1204

Background: Randomized controlled trials are the gold standard for comparative effectiveness research (CER), but may not fully reflect real-world (RW) clinical settings due to stringent eligibility criteria and treatment schedules. RW-CER enables treatment comparisons in populations seen in clinical practice, but remains challenging due to bias and confounding. A systematic framework to assess feasibility and conceptualize the design of RW-CER studies would help to overcome these limitations.

Objectives: To develop and test a framework to assess the feasibility of RW-CER and promote rigorous study design that adequately controls for bias and confounding.

Methods: This framework for RW-CER consists of a series of A) feasibility and B) pre-design steps. The feasibility steps are: A1) assess adequate capture of exposures, outcomes, and covariates; A2) identify potential confounding factors and effect modification; A3) assess comparability of treatment populations to understand the overlap in baseline characteristics between treatment groups. The pre-design steps are: B1) assess the no unmeasured confounding assumption; B2) initially evaluate potential treatment effects; B3) establish a minimum treatment effect size and assess sample size. As an example of its practical application, the framework was used to assess the feasibility and inform the design of a series of RW-CER studies in patients with chronic obstructive pulmonary disease (COPD).

Results: The framework holistically evaluates proposed RW-CER studies using an adaptation of the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) framework, with each step in the feasibility pathway taking the place of the GRADE outcomes. Each step is assessed against four criteria: study limitations (risk of bias), imprecision, inconsistency of results, and indirectness of evidence. For each step, a risk score (scale 0–2) is given for each of the four criteria leading to an overall suitability score to proceed to CER of 0–8 (0–2: high; 3–4: moderate; 5–6: low; 7–8: very low). The outcomes of the feasibility assessment in conjunction with the pre-design phase aided optimal study design to help overcome any perceived limitations in the assessed RW-CER studies in COPD.

Conclusions: This framework will help overcome the challenges of RW-CER by providing a fully transparent assessment, supporting or not supporting the decision to proceed with RW-CER studies, and identifying limitations to be addressed through study design. Practical application of this framework has been demonstrated in RW-CER studies in COPD.

Funding: GSK (214731)