Background: To assist their users in making informed decisions about what treatments to use, BMJ Clinical Evidence devised a categorisation system, which aimed to identify treatments that work (benefits outweigh the harms) and highlight treatments that do not work (harms outweigh benefits). However, in 2017, the ‘state of the evidence’ for the around 3000 treatments assessed by Clinical Evidence using randomised-controlled trial (RCT) evidence suggested that around 50% of treatments were categorised as ‘Unknown effectiveness’ for specific indications. Cochrane Clinical Answers (CCAs) also aims to inform decision making by making Cochrane review evidence more accessible and actionable, and faces similar challenges regarding uncertainty.
Objectives: To assess the ‘state of evidence’ for treatments assessed in 1000 CCAs, using a similar categorisation to that devised by BMJ Clinical Evidence, in particular focusing on highlighting the proportion of CCAs affected by insufficient RCT data.
Methods: An assessment of 1000 CCAs covering a wide range of clinical disciplines, including Cardiology, ENT disorders, Emergency Care, Mental health and Pregnancy & Childbirth, was performed. Each answer was categorised as to whether it provided guidance to: ‘use treatment’, ‘use treatment but some caveats’, ‘do not use treatment’, or ‘treatment effectiveness unknown’.
Results: Assessment of 1000 CCAs suggests some parity with the results of the BMJ Clinical Evidence, with 18% of CCAs giving guidance to ‘use treatment’, 35% suggesting ‘use treatment but some caveats’ (as to how/when to use, need to balance benefits and harms, or doubts about the strength of the evidence), 9% suggesting ‘avoid use’, and 38% treatment effectiveness unknown.
Conclusions: CCAs are a great tool to filter the vast amount of data from Cochrane reviews and the RCTs they summarise to make it easier for healthcare professionals to apply high-quality evidence when managing patients. However, there are many questions for which we do not have a clear answers where the main strength of CCAs is to quickly highlight that clinicians need to apply expert judgement and non-randomised evidence.