Michael G. DeGroote School of Medicine, McMaster University
Introduction: Large number of systematic reviews (SRs) are published each year. While often of poor methodological quality, they are placed at the top of the ‘hierarchy of evidence’ and frequently cited. We performed this study to provide a longitudinal analysis of the methodological quality of SRs published in major urology journals over a 6-year time-period (2016-2021). Methods: We systematically searched and analyzed all SRs related to questions of therapy or prevention published in the five major urology journals (European Urology, Journal of Urology, BJU International, Urology and World Journal of Urology). Three independent reviewers working in pairs selected eligible studies and abstracted the data in duplicate. We used the updated Assessment of Multiple Systematic Reviews (AMSTAR-2) instrument to assess SR quality. We performed pre-planned statistical hypothesis testing by time-period and journal of publication in SPSS Version 27.0. Results: After screening 740 references (latest search date: 3/25/2022) we included 258 SRs of which 144 (55.8%) were published from 2016-2018 and 114 (44.2%) from 2019-2021. The main contributors were European Urology (30.2%), World Journal of Urology (22.1%) and Urology (20.5%). The most frequent topic by far was oncology (45.3%), followed by voiding dysfunction (20.5%) and infection/inflammation (8.9%). Overall, 2.3%, 3.5%, 18.6% and 75.6% of SRs had high, moderate, low, or critically low confidence ratings, respectively. ‘Justification for excluding individual studies’ and ‘protocol registered before commencement of the review’ were the main critical AMSTAR-2 criteria not met by 89.1% and 53.5% of SRs, respectively. Comparing time-periods 2016-2018 to 2019-2021, the proportion of SRs with high/moderate ratings did not increase (4.9% versus 6.1%; p = 0.481). The proportion of high/moderate quality reviews was highest for BJU International (15.4%) followed by Journal of Urology (9.7%), World Journal of Urology (5.3%), Urology (1.9%) and European Urology (1.3%; p = 0.030). Conclusions: Most SRs published in the urological literature continue to have serious methodological limitations and cannot be relied upon. There is a critical need for raising greater awareness among SR authors, editors, and consumers of the urological literature about established standards for high-quality SRs in order to improve their future quality. SOURCE OF Funding: Departmental.
