(152) A Step-By-Step Guide for Causal Study Design When Estimating Treatment Effects Using Real-World Data

Publication Number: 1197

Background: Causal inference (CI) is an innate part of scientific research. While inherently challenging in observational data settings,CI in observational research is growing more important, driven by the need for generalizable and rapidly delivered real-world evidence (RWE) to inform regulatory, payer, and patient/provider decision-making. CI with observational data combines numerous theoretical and technical concepts , necessitating specialized training and competency. Existing methodological literature on this topic is rich but can be complex and daunting to navigate.

Objectives: To develop a step-by-step guide to help researchers conduct high-quality CI studies using observational data.

Methods: A team with diverse training and research backgrounds developed a visual step-by-step guide to causal study design and a corresponding glossary, after a comprehensive review of CI literature. The guide addresses key conceptual issues that researchers should be aware of to develop confidence in implementing CI methods in observational research.

Results: We describe eight steps as a guide to reseachers in designing causal studies with observational data. Step 1: Define an explicit causal research question. Step 2: Decide on a “first treatment carried forward” (analogous to “intention to treat”) or an “as-treated” (analogous to “per protocol”) outlook. Step 3: Define the population for the counterfactual contrast, e.g., average treatment effect in the entire study population or in the treated. Step 4: Select a measure of effect, e.g., rate or risk, difference or ratio. Together, steps 2-4 define the study estimand. Step 5: Develop a directed acyclic graph for the research question and estimand to clarify the causal pathway. Step 6: Identify potential biases, e.g., immortal time, healthy adherer effect. Step 7: Select appropriate design elements and statistical methods to evaluate and address potential biases. Step 8: Conduct sensitivity analyses to assess the robustness of the methods used and quantify remaining biases, e.g., unmeasured confounding.

Conclusions: We outlined steps and described key conceptual issues of importance in designing CI studies, and created a visually appealing, user-friendly resource to help researchers clearly define and navigate these issues. This guide serves to enhance the quality, and thus the impact, of RWE from observational research.