What Do Real World Data Validation Best Practices Look Like? Operationalizing Guidance for Real-world Studies Intended for Decision-making

Background: Guidance for decision-grade real world evidence (RWE) studies emphasizes the importance of validating the definitions of key study variables to facilitate robust estimation of treatment effects. Often, complete verification (where the accurate value of a variable is assigned to every study patient based on a reference standard) is infeasible, and other approaches must be used to demonstrate sufficient validity of measures and instill confidence in the interpretation of results.

Objectives: 1. Provide an understanding of approaches to assess and improve the validity of real-world variable definitions when complete verification is infeasible, and in what circumstances they are most applicable
2. Concretize the application of these approaches using examples and share lessons learned
3. Begin a discussion of the rationale and evidence decision-makers may need to increase trust in proxies for complete verification

Researchers, regulators, health technology assessors, and payers using RWE to support decision making would benefit from attending this symposium.

Description: This symposium will explore approaches to assess and improve the validity of variable definitions, including those for exposures, outcomes, entry criteria, and confounders, when a ‘gold standard' is wholly unavailable. Examples are adjudication, natural language processing- or machine learning-derived variables, and comparisons to internal/external benchmarks. Illustrative cases will demonstrate application of these approaches.

●

Introduction: The first section will provide a brief overview of regulatory guidance and best practices for validating measures, including sources of "truth", assessment of performance, and other aspects. (10 min — Ulka Campbell)

● Sponsor endpoint validation experience: The second section will describe an effort by Lilly to establish a framework for developing solid evidentiary support of endpoint validity. (15 min — Sangmi Kim)

● Sentinel validation experience: The third section will describe validation approaches used within FDA's Sentinel program. (15 min — Darren Toh)

● Health technology assessment (HTA) experience: The fourth section will focus on expectations regarding evidence of validated definitions for reimbursement and coverage decisions. (15 min — Seamus Kent)

●

Conclusions: The last section will provide final remarks on validation principles and lessons learned. (10 min — Jeff Brown)

● Q&A/

Discussion: This will be an interactive session to address audience questions and discuss evidentiary standards for decision-making when complete verification is infeasible. (25 min — facilitated by Jeff Brown, all panelists participate, Carla Rodriguez-Watson to join discussion)