The agreement of treatment-effect estimates from rapid reviews using abbreviated literature searches and traditional Cochrane reviews: A meta-epidemiological study

ID: 

2009

Session: 

Poster session 2 Thursday: Evidence synthesis - methods / improving conduct and reporting

Date: 

Thursday 14 September 2017 - 12:30 to 14:00

Location: 

Exhibition Halls 1 & 2

All authors in correct order:

Ladanie A1, Klerings I2, Wagner G2, Heise TL3, Dobrescu AI4, Armijo-Olivo S5, Stratil JM6, Lhachimi SK3, van Noord M7, Mittermayr T8, Gartlehner G9, Nussbaumer-Streit B10, Hemkens LG11
1 Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel & Swiss Tropical and Public Health Institute, University of Basel, Basel, Switzerland
2 Department of Evidence-based Medicine and Clinical Epidemiology, Krems, Austria
3 Leibniz Institute for Prevention Research and Epidemiology – BIPS, University Bremen, Germany
4 Genetics Department, Victor Babes University of Medicine and Pharmacy Timisoara, Romania
5 Faculty of Rehabilitation Medicine, University of Alberta & Institute of Health Economics, Edmonton, Alberta, Canada
6 Institute for Medical Informatics, Biometry and Epidemiology, Ludwig-Maximilians-University, Munich, Germany
7 Medical Center Library & Archives, School of Medicine, Duke University, USA
8 Ludwig Boltzmann Institute for Health Technology Assessment, Austria
9 Cochrane Austria, Danube University Krems, Krems & RTI International, NC, Austria & USA
10 Cochrane Austria, Danube University Krems, Krems, Austria
11 Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, University of Basel, Basel, Switzerland
Presenting author and contact person

Presenting author:

Aviv Ladanie

Contact person:

Barbara Nussbaumer-Streit
Lars Hemkens
Abstract text
Background: Rapid reviews have become a pragmatic alternative to traditional systematic reviews by streamlining methodological processes such as literature searches, aiming for faster provision of decision-relevant information. The agreement of effect estimates obtained from abbreviated literature searches as conducted in rapid reviews and those obtained from extensive searches as performed in Cochrane or Campbell reviews is unknown.
Objectives: To assess differences between treatment effects estimated from abbreviated and extensive literature searches.
Methods: We selected 44 Cochrane reviews published between 2012 and 2016 on diverse topics that had a binary outcome in the main summary of findings table (a subset of randomly selected reviews used for a related project [1]). We constructed 14 variants of abbreviated searches based on the original CSRs search strategy by combining searches in MEDLINE, Embase, or CENTRAL with or without searching of reference lists. From the first binary outcome’s meta-analysis of each CSR, we extracted trial results, recorded for each trial which abbreviated search variant identified it, and calculated a summary treatment effect estimate per search variant (summary odds ratio, sOR) including only trials identified by that variant using random-effects meta-analyses. We then determined how often sORs of abbreviated and original searches differed in the direction, in the inclusion of the null (i.e. nominal statistical significance), and beyond chance (i.e. the ratio of sORs excluded the null). We follow an epidemiological approach to evaluate the overall relationship of treatment effect sizes derived by abbreviated and original searches in an overarching meta-analysis across all 44 ratio of sORs.
Results: Analyses are ongoing and results will be available at time of the Summit.
Conclusions: Our results will systematically quantify the impact of faster and abbreviated searching on treatment effect estimates across a wide range of CSR topics.

[1] Nussbaumer-Streit et al. Assessing the validity of abbreviated literature searches for rapid reviews: protocol of a non-inferiority and meta-epidemiologic study. Sys rev. 2016;5:197