Background: In many reviews quantitative data are synthesised narratively: we estimate that narrative synthesis (NS) is used in 20% of Cochrane Reviews. A key criticism of NS is lack of transparency, and the risk of subjective assessments of evidence. This makes it difficult to assess rigour and potential bias in NS. Developments to improve synthesis methods have largely overlooked NS of quantitative data.
Objectives: 1) to establish current practice in reporting of NS of quantitative data; and, 2) to develop resources to improve the implementation of NS of quantitative data.
Methods: Using a random sample of systematic reviews from the McMaster HealthEvidence database, we assessed 75 public-health reviews that had used NS for their key outcome(s). Data were extracted on: reporting and justification of NS methods; management of heterogeneity; and, transparent links between the data and text.
Results: Description of NS methods and reference to NS guidance was absent in most reviews (95% n=71/75). Investigation and management of heterogeneity was typically unclear. In 41% (n=31) of reviews, limited presentation of tabulated data prevented transparency between the data and synthesis findings. Grouping of studies was used to manage heterogeneity but with no explanation, and heterogeneity in effects was rarely investigated. A 2-year programme, funded by the Cochrane Strategic Methods Fund (SMF), to develop resources to facilitate improved implementation of NS is now under way. The resources being developed include: consensus-based reporting guidelines; guidance for authors on implementation; and, online training.
Conclusions: These findings support the criticism that NS is characterised by a lack of transparency, raising concern about the potential for bias in a large volume of the SR evidence base. The near absence of description of NS methods in reviews suggests a lack of clarity among authors about NS as a method. The work under way aims to address this by providing clear guidance on reporting and implementation of NS. These resources will provide support for authors undertaking NS, as well as those assessing the adequacy of NS within Cochrane and beyond.