Reporting of the Identification of Studies Through Other Means in Evidence Synthesis: A Cross Sectional Study

Objectives: Recent reporting guidelines have established higher standards for the reporting of search strategies in evidence synthesis projects. In reviews compliant with PRISMA 2020, readers can in principle calculate how many included studies were identified via “other methods” (than database/registry searching). We will study reviews which appear to comply with PRISMA 2020 to find out what proportion of the included studies in those reviews were identified through other methods, and whether and how the authors of those reviews discussed the implications of a low or high proportion of included studies having been identified through other methods.

Methods: We used Citation Chaser to identify more than three thousand unique articles in the Lens database which cite the preprint or VoR of PRISMA 2020 or the PRISMA 2020 E&E paper. We screened these articles to find review papers with PRISMA 2020-compliant flowcharts. We conducted a pilot study by extracting data from 100 reviews:
-Stated review type
-Stated PRISMA-S compliance
-Number of included papers in total, from the traditional searching branch, and from the other methods branch
For reviews which mention other methods, we also extracted:
-Which other methods were reported (using PRISMA-S categories: online resources and browsing, citation searching, contacts, and other methods)
-Any discussion by the authors of the implications of the yield of their other methods branch

Results: Our results will be shared via an OSF project as well as the conference paper. We found that most papers using PRISMA 2020 are systematic reviews and meta-analyses. Many authors state that they are using PRISMA 2020 (inappropriately) as a methods guideline. Only a small number of PRISMA 2020-style flowcharts include an other means column. The proportion of included studies identified through other means (when other means were attempted and details were reported) varied from 3% to 40%. The yield of "other methods" searches was not discussed by authors.

Conclusions: The results of our study will be useful for librarians helping evidence synthesis authors comply with PRISMA 2020 and PRISMA-S. Our results may also be useful to evidence synthesis authors deciding whether to use other methods as part of their study identification process. Finally, our results suggest that the development of a potential search quality indicator that does not require librarian-level search literacy, based on the yield of a review paper’s other identification methods, is a long way off.