How do characteristics, reporting methods and preparation times differ between systematic reviews with and without a published protocol?

ID: 

18142

Session: 

Long oral session 8: Methods for overviews

Date: 

Wednesday 13 September 2017 - 16:00 to 17:30

Location: 

Meeting Room 1.63

All authors in correct order:

Allers K1, Hoffmann F1, Mathes T2, Pieper D2
1 Department of Health Services Research, School of Medicine and Health Sciences, Carl von Ossietzky University Oldenburg, Oldenburg, Germany
2 Institute for Research in Operative Medicine, Faculty of Health - School of Medicine, Witten/Herdecke University, Cologne, Germany
Presenting author and contact person

Presenting author:

Katharina Allers

Contact person:

Katharina Allers
Abstract text
Background: Preparing a protocol before publishing a systematic review (SR) can minimise the potential for bias, increase transparency and reduce duplication. Despite the advocated importance and the potential advantages of protocols for SRs, there is currently no analysis that compares SRs with a protocol to SRs without a protocol.

Objectives: To explore trends in published protocols of SRs and to analyse how SRs with a published protocol differ from those without a published protocol.

Methods: We searched PubMed to 31 December 2016 to identify protocols of SRs. For all protocols published in 2012 and 2013, the respective SR was searched. For each of these SRs we matched a SR without a published protocol controlling for publication year and journal.

Results: The number of published protocols increased from 42 in 2012 to 404 in 2016. One hundred and twenty five (125) protocols were published in 2012 and 2013. About one third of SRs are still not published after 3-5 years. We included 80 SRs and 80 control SRs in our analysis. SRs with a published protocol are more transparently reported than their controls and are completed with more effort (e.g. higher number of databases searched and more languages considered). Moreover, risk of bias was assessed more frequently in SRs with a published protocol than in controls (86.3% vs. 60.0%; p=0.0002). However, the median time from search to submission was much longer for SRs with a published protocol (325 vs. 122 days; p=0.0009) and more than half of the SRs with a published protocol had performed the final search before submitting the protocol for publication. Almost two thirds of the SRs with a published protocol and about 10% of those without are registered in PROSPERO. Of these, only 22.2% have the updated status ‘published’.

Conclusions: Quality, transparency and currency are cornerstones of SRs. However, none of them should be achieved at the expense of the others. The large number of unpublished SRs after 3-5 years is alarming. Updating the status of a SR in PROSPERO should be used more often. Based on our results, we suggest critically discussing the current practice of publishing protocols of SRs.