Evaluating a Current Awareness Search Service

Background: For several years librarians in our VA Library Network (VALNET) have provided current awareness searches on healthcare topics of interest to the agency. All VHA employees may elect to subscribe to these alerts. Depending on the volume of literature, PubMed search results are filtered and emailed on a weekly, bi-weekly, or monthly basis to subscribers.

We initiated this project to develop an evaluation of the service to determine the level of satisfaction, relevancy, and perceived impact of the subject searches. Our goal was to assess whether the search service provides value for subscribers and if it should be continued and improved.

Description: A 9-question electronic survey was developed and tested in 2019. Subscribers to three alert topics were surveyed each month between January 2020 and January 2021. Each survey was open for one week. A total of 12,976 subscribers to 34 topic searches received surveys. Respondents were asked to self-identify from a list of health professions. Other questions asked subscribers about their level of satisfaction with the service, whether the alerts provided relevant information, and if the alerts had a positive impact on their work. Results of individual surveys were compiled and analyzed, including free text comments on 2 of the 9 questions.

Conclusion: Respondents indicated that the alerts were relevant to the subject and the participants’ interests (97%) and that they significantly impacted the ability to perform their work (68%). Satisfaction with the alert service was rated good to excellent by 92% of respondents. Subscribers’ comments provided useful suggestions to improve the service. When subscribers requested an easy option to cancel their subscriptions, an updated cover message was added with an opt-out link. Comments on another alert complained about the ‘overwhelming’ number of citations sent with the alerts, prompting a review and revision of the search strategy.