Resident physician Department of Surgery, Queen's University, Ontario, Canada
Objectives: The objective of this study was to determine the accuracy and performance of commonly-used electronic methods for de-deduplicating references retrieved from searching multiple bibliographic databases.
Methods: A heterogeneous sample of references was obtained by conducting a similar topical search in MEDLINE, Embase, PsycINFO and Cochrane CENTRAL. Manual abstraction was used to identify duplicate references among the search results and develop a gold standard for comparison. De-duplication methods included Ovid multifile search, EndNote desktop (versions X9 and 20), Mendeley, Zotero, Covidence and Rayyan. False negative and false positive duplicate references for each method were identified and then the accuracy, sensitivity, and specificity of each was calculated.
Results: Default de-duplication settings in Ovid multifile search and dedicated review software (Covidence and Rayyan) significantly outperformed reference management software. Ovid, Covidence and EndNote 20 possessed the highest specificity (1.00, 1.00 and 0.998, respectively) for identifying duplicate references, while Rayyan demonstrated the highest sensitivity (0.96). The accuracy of EndNote desktop improved from the X9 version (0.76) to the newer 20 version ( 0.92).
Conclusions: This study highlights the strengths and weaknesses of commonly-used electronic methods for de-duplicating references and suggests strategies to avoid unintentionally removing eligible studies and introducing bias into syntheses. When selecting database platforms and utilizing software programs for the review process, de-duplication functionality and performance is an important consideration for improving the efficiency and quality of evidence syntheses.