Creation of an Institutional Quality Assessment Tool Repository

Background: Our library's comprehensive literature search service primarily focuses on expert searching and guidance on general evidence synthesis methodology. We have experienced an increase in the number of questions about quality assessment tools. Our practice is to suggest the most commonly used tools. However, teams sometimes reported these tools did not meet their needs. To solve this problem, we created a repository and finding aid of quality assessment and risk of bias tools. This allowed us to become more familiar with the array of tools available, along with their purposes and features, and ultimately was helpful for our research teams.

Description: To construct our repository we conducted a literature search for quality assessment and risk of bias tools. We began compiling a list that provided the tool's name, description, citation, intended use, whether it was validated, and any notable details we discovered when researching the tool. We included tools whose purpose was to evaluate the methodological quality or risk of bias of study designs. In most cases, we omitted critical appraisal tools, which we defined as anything that was designed solely to be used by clinicians or students to evaluate the quality of an article for class work or patient care. Our final product is a comprehensive, searchable table that is freely available on our systematic review services guide. 

Conclusion: There are a wide variety of quality assessment and risk of bias tools available to meet most study designs. Reporting guidelines such as STROBE, which are designed to aid in the reporting of research, and not specifically for quality assessment, were not added to the repository. It was frequently difficult to determine if a tool was validated, though many common tools have had their interrater reliability tested. Where possible, we prioritized gathering risk of bias tools as per Cochrane Handbook's recommendations. We included notes that added additional information about usability or purpose.

The next planned step is to gather qualitative feedback from our researchers about their experience with the individual tools.

Having a single searchable resource that captures the characteristics of a variety of tools can be a helpful for librarians or researchers conducting a systematic review outside their field of practice.