Selective searching for high-quality health-related evidence syntheses – more bias or time gained?

ID: 

2088

Session: 

Poster session 2 Thursday: Evidence synthesis - methods / improving conduct and reporting

Date: 

Thursday 14 September 2017 - 12:30 to 14:00

Location: 

Exhibition Halls 1 & 2

All authors in correct order:

Metzendorf M1, Richter B1
1 Cochrane Metabolic and Endocrine Disorders Group, Germany
Presenting author and contact person

Presenting author:

Maria-Inti Metzendorf

Contact person:

Maria-Inti Metzendorf
Abstract text
Background: Cochrane and other organisations producing health-related evidence syntheses advocate using a comprehensive search approach consisting of sensitive search strategies conducted in multiple databases. This results in resource-intensive and time-consuming search and screening processes. Findings of two recent publications suggest that further exploration of the current recommendation is warranted.

Objectives: We aimed to reproduce and complement the findings by Halladay 2015 (1) and Hartling 2016 (2) by investigating a third dataset of Cochrane Reviews regarding the impact of non PubMed-indexed publications on the overall results.

Methods: We included all reviews (n = 47) produced in the last 5 years by the Cochrane Metabolic and Endocrine Disorders Group and retrospectively checked the origin of studies used (not only assessing included studies, but also those ongoing and awaiting assessment). We analysed whether the restriction to publications indexed in PubMed resulted in relevant changes to the effect measures. The reviews were classified by population and type of intervention.

Results: We extracted 1037 publications from 47 reviews. 89% of publications were available in PubMed. An analysis of those reviews using more than 1/3 of publications not included in PubMed showed that these were reviews on complementary medicine and dietary supplements. After excluding all reviews on both of these topics 92% of publications were available in PubMed. We are currently calculating the effects on the meta-analyses and will present results at the Summit.

Conclusions: Preliminary evidence suggests that selective searching produces less bias than previously assumed, but primarily relates to reviews evaluating therapeutic interventions and focussing on clinical trials. It probably does not apply to all types of therapeutic interventions. Further research is needed to confirm in which topics a selective literature search can be a sound practice and whether this approach is suitable for clinical practice guidelines as well.

1. Halladay et al. Clin Epidemiol. 2015 Sep;68(9):1076-84.
2. Hartling et al. BMC Med Res Methodol. 2016 Sep 26;16(1):127.