Background: In several disciplines, systematic reviews are becoming an ‘industry standard’ for evidence synthesis. Many stakeholders in the field of medicine (including publishers, authors, commissioners and decision makers) are aware of the key principles of systematic review. In other fields, such as environmental sciences, however, systematic reviews are a relatively novel methodology, and whilst there is a general appreciation for the added value that a systematic review has over a traditional literature review, broadly speaking there is only a limited understanding of the necessary steps and safeguards needed to ensure a systematic review is truly reliable. As the field of evidence synthesis in environmental science continues to develop we face an increasing number of reviews that claim to be ‘systematic reviews’ despite lacking transparency and repeatability, performing little critical appraisal, failing to attempt comprehensiveness and often performing vote-counting.

Objectives: Here we present examples of poor quality and unreliable reviews that claim to be systematic reviews from the field of environmental management and conservation.

Methods: We have collated examples of syntheses that refer to themselves as ‘systematic reviews’ and select examples to demonstrate common limitations and sources of potential bias.

Results: We highlight commonly seen misconceptions regarding systematic reviews and map and the risks of the dilution effect of these sub-standard reviews. In particular, we use a key example of a recent review from the field of biodiversity and climate change to express our concerns.

Conclusions: We call for a concerted effort across disciplines to ensure that the standard of true systematic reviews remain high. Systematic reviews could be adversely affected by limited appreciation of the required rigour, transparency, repeatability and comprehensiveness required to conduct a systematic review.