Background: Syntheses of syntheses, otherwise known as overviews, have become an increasingly prevalent approach to synthesising research. Although overviews are becoming prevalent, they are a relatively nascent and undeveloped synthesis method that pose unique methodological challenges. While significant empirical work has been undertaken to inform and improve systematic review methods, limited research and guidance is available for overviews.

Objectives: To examine the methodological quality of overviews and provide guidance for the conduct and reporting of overviews.

Methods: A systematic search for overviews that aimed to synthesise more than one empirical education-related review was conducted in multiple online databases and grey literature repositories. Two authors independently screened and selected studies and extracted data using a standardised codebook. Studies were analysed descriptively; we aimed to elucidate all aspects of the overviews and calculate the proportion of characteristics reported across 17 different methodological characteristics. We also compared methodological quality and reporting of recent overviews to early overviews using t-tests.

Results: Twenty-five overviews met eligibility criteria for this study. Our analysis revealed that many commonly reported aspects of systematic reviews were regularly unreported. Of the 25 overviews included, 11 used a narrative synthesis technique (44%) whereas 14 used a quantitative analytic technique (56%). Overview authors rarely took into account overlap between reviews or up-to-datedness of reviews.

Conclusions: Overviews offer an exciting, yet challenging new method for synthesising and managing the ever-expanding volume of research. Given the issues and limitations identified in this study, care must be taken in interpreting and using extant overviews. Moreover, to ensure the validity and utility of overviews to inform practice and policy, it is important that the conduct and reporting of overviews improve. As a result of this study, we developed conduct and reporting guidelines for overviews which will be presented and discussed.