It is important to assess the impact of an educational activity on levels of knowledge and competence not only to determine the success of the activity, but also to discover if any educational gaps remain. Moore’s Levels for assessing outcomes in education are well established and the majority of Independent and Continuing Medical Education activities are measured using this framework. Moore’s Levels 3 (knowledge) and Level 4 (competence) are currently measured as standard and while these are very informative, they can only hint at what might happen in daily clinical practice. To really understand the impact of education in the real-world, Level 5 outcomes (performance), need to be assessed. However, Level 5 can be challenging to measure as ideally it would involve direct observations of performance in the clinical setting. This is not usually feasible for educational activities that reach a large audience. So how can education providers assess Level 5 outcomes?
Self-reported performance can be used, but this is subjective and so carries inherent biases and limitations. One alternative solution may be a multi-pronged approach that incorporates a short questionnaire, review of patient records and artificial intelligence to validate responses. The questionnaire is fielded pre- and post-education and the questions are asked neutrally, with no incorrect answer. Redacted patient records are collected before and after HCPs participation in the education to give direct insights into changes. The artificial intelligence uses facial recognition software to compare emotional and stated responses to further validate whether HCPs truly intend to change their practice.
There are challenges with this multi-pronged approach; commitment is needed from the participating HCPs, time is needed for the assessment processes and there is a cost burden.
Assessment of Level 5 outcomes is becoming increasingly important and education providers need to continue to develop innovative approaches to their measurement, and it will be interesting to observe the different solutions that emerge.
In this session, we will use an interview format to explore the benefits and challenges of this multipronged approach to assessment of Level 5 outcomes and consider future solutions.
Innovation: The artificial intelligence uses facial recognition software to compare emotional and stated responses to further validate whether HCPs truly intend to change their practice.