Developing a Tool to Evaluate Aggregated Outcomes Data for Pharmacist Educational Initiatives Related to Influenza Prevention

According to the CDC, all individuals aged 6 months and older should receive an influenza vaccine; however, only about 50% of eligible Americans receive the vaccine annually. Pharmacists across multiple practice settings are responsible for answering questions related to vaccination as well as administering influenza vaccines within their communities. Additionally, as the influenza virus and components of the vaccine change from year to year, education is essential to ensure pharmacists are comfortable providing education and recommendations to the public on vaccine eligibility, adverse effects, and vaccine hesitancy. Strategies to close these identified gaps undertaken by Pharmacy Times Continuing Education™ between 2019 and 2022 focused on providing live education that incorporated audience response to engage participants in active learning, including presentation of patient cases and opportunity for questions and answers with faculty. The outcomes results including pre/posttest data, confidence, and intended practice changes across these programs will be aggregated. Aggregated data will provide insight into the effectiveness of these educational activities as a whole, including trends among the different activities, and help to identify remaining gaps that need to be addressed when designing future continuing education on this important topic.

Educational Strategy:

Between 2019 and 2022, Pharmacy Times Continuing Education™ developed and delivered 5 distinct educational programs for pharmacists related to influenza vaccination. All 5 programs had live data and 3 of the 5 had on-demand data available for analysis. The outcomes for these projects were evaluated and the data were extracted and collected on a data collection form template created by the quality improvement (QI) working group. Data extracted assessed pharmacist knowledge, attitudes, and confidence and included pre/post assessment and confidence questions as well as demographics for live and enduring programs. Demographic data included patient interactions per year, years of practice, and primary practice setting. Additional data included Moore’s level 2 outcomes, such as satisfaction with the quality of the activity and faculty (both reported as percent who rated activity as good or excellent), as well as perception that learning objectives were met (reported as percent who rated activity as average to excellent). Data were aggregated to show the averages of assessment questions for each program and also for the overall therapeutic area. Assessment questions were aggregated by question category, which was also developed by the QI working group and reported as an absolute difference. Confidence questions data were aggregated and reported as the percentage of participants reporting moderately, very, or extremely confident before and after the program. Finally, the top 3 intended practice changes were aggregated to determine the potential impact on patient care.

Patient-Level Outcome(s) Measured:

To date, 22,027 learners participated in the activities. Average years in practice were 27.5% for fewer than 5 years, 14.4% for 6-10 years, and 52.9% for greater than 10 years. The majority of learners were in the practice setting of retail/community (48.5%). The average increase in knowledge with the pre/post assessment questions was 20%. Although the change was similar for learners who participated in the live activities compared with the enduring activities (21% vs 19%), the live participants had a much higher baseline knowledge (45% vs 35%). Questions pertaining to the categories of pharmacotherapy and treatment recommendations/guidelines had the highest growth in knowledge and questions on epidemiology and communication had the smallest increase in knowledge. The learners who rated themselves moderately, very, or extremely confident increased 26%. However, learners in live activities noted a higher increase (33%) compared with those participating in on-demand activities (20%).

The practice changes with the highest responses were within the categories of addressing vaccine hesitancy (55%), improving or increasing patient communication (48%), and identifying appropriate patients for vaccination (45%). Interestingly, the highest reported barriers were these same categories, demonstrating the need for future programs to address strategies aimed to overcome these barriers.

These results provide insight into areas where the baseline knowledge and confidence of pharmacists may be strong or lacking as well as barriers they may face that will help us tailor learning objectives of future programs to meet the needs of our pharmacists.