(279) Definition of Outcomes and Success Criteria for Healthcare Professional or Patient Surveys Designed to Assess the Effectiveness of Risk Minimization Measures (eRMM)

Publication Number: 953

Background: Post-authorisation safety studies to assess the effectiveness of risk minimization measures (eRMM) are commonly required by the European Medicine Agency’s Pharmacovigilance Risk Assessment Committee (PRAC) when educational material on the risks of a medicinal product is required to be distributed to healthcare professionals (HCPs) or patients. Currently there is no clear guidance on how to define outcomes and success criteria in eRMM studies based on HCP and patient surveys.

Objectives: Our aim was to review different ways of defining outcomes in eRMM survey studies and share our lessons learned to approach success criteria depending on the situation.

Methods: We reviewed four eRMM survey study documents (2 protocols, 1 SAP and 1 report) we recently developed and their related PRAC feedback and synthesized learnings from these with a focus on defining outcomes and success criteria.

Results: Outcome definition: eRMM surveys are often short and, in the authors’ experience, the expectation of the PRAC is that each question will correspond to a variable, and responses to each variable will be presented separately in the results. Outcomes commonly cover 3 categories: receipt/awareness, knowledge/understanding, and self-reported behaviour. The importance of these categories varies depending on the product and its risk profile and, with the objective of minimizing risks, one might want to focus on specific questions, e.g., knowledge of an important risk. In three studies, we proposed a hierarchy of questions via creation of composite outcomes (predefined group of core questions to which success criteria would apply). Composite scores were defined based on one outcome category or spanning all 3 categories. The endorsement of this approach by the PRAC was study-dependent, and what was accepted for one study was not necessarily endorsed for another, depending potentially on the type and level of the risk to be minimized.
Success criteria: Success is commonly defined as ≥80% correct responses; however, the dilemma is whether this should be determined as ≥80% of participants providing correct responses to specific questions, or as participants responding correctly to ≥80% of all questions on average. Our experience showed that the former is more commonly used to determine success criteria. In circumstances where it is particularly important that a risk is well understood, additional thresholds (e.g., 90%, 100%) can be added.

Conclusions: Our experience is that eRMM survey outcomes can be defined in different ways, according to the product and focus of the educational material, while the definition of success criteria across studies is more homogeneous. More guidance from the PRAC is needed to expedite protocol approvals.