Background: The exponential growth of biomedical literature has greatly increased the burden on those producing systematic reviews and guidelines, thus hindering our ability to practice evidence-based medicine. To meet these demands, we need new computational tools and methods to expedite evidence synthesis.

Objectives: RobotReviewer aims to automate, or semi-automate the task of data extraction for evidence synthesis. The system extracts (an increasing number of) key variables from full-text articles (PDFs) describing the conduct and results of randomised-controlled trials (RCTs).

Methods: RobotReviewer incorporates a number of novel machine-learning (ML) models, which have been trained on large annotated datasets (currently including the Cochrane Database of Systematic Reviews, and the Cochrane Crowd EMBASE set). The web tool takes a set of RCTs as input, and produces a downloadable summary report (see Figure). The tool incorporates a PDF viewer to allow the user to see and interact with extracted text in context.

Results: Currently, RobotReviewer can: (1) describe the study design (e.g. RCT or not); (2) identify sentences that describe the trial population, interventions/comparators and outcomes; and, (3) assess biases using the Cochrane Risk-of-Bias tool (both judging whether at low or high/unclear risk of bias, and identifying text justifying the judgment). In future, we aim to continue to improve upon individual task accuracy, and extend the system to extract the full range of variables needed for evidence synthesis. We have released the software and trained ML models as open source (under the GPL v3.0 license) on our project website (http://www.robotreviewer.net/) together with a live demonstration. RobotReviewer also features a REST API, which enables the underlying annotation models to be incorporated into other software systems.

Conclusions: RobotReviewer represents a step toward more efficient evidence synthesis via automation. Adopting such technologies is critical if future evidence syntheses are to remain timely and comprehensive.