Background: Non-randomised studies provide evidence about adverse effects and long-term outcomes and are often included in systematic reviews about healthcare interventions. The ROBINS-I tool (Risk Of Bias In Non-randomised Studies of Interventions) facilitates an evaluation of risk of bias in these studies. The tool covers bias due to confounding, selection of participants into the study, classification of interventions, deviations from intended interventions, missing data, measurement of outcomes, and selection of the reported result. Although we believe that core bias domains are covered by the tool, ROBINS-I was developed primarily on epidemiological principles and expert opinion, rather than literature review.

Objectives: To collate the large number of biases described in the literature and determine whether the ROBINS-I tool comprehensively captures these biases.

Methods: We searched Medline, Embase, Web of Science, the University of Bristol library collection and Amazon books, for papers and textbooks that listed definitions of biases in epidemiological research. We included papers and textbooks that listed at least 10 biases. To organise the definitions of biases, we constructed directed acyclic graphs (DAGs) and grouped biases with a common causal structure. We drafted definitions for each unique type of bias. An expert panel approved all DAGs and definitions. For biases that are relevant to non-randomised studies of interventions, we considered whether each was covered by ROBINS-I.

Results: We included 22 papers and 17 textbooks, which described 239 biases. Ambiguous definitions made classification difficult; however, the constructed DAGs helped us differentiate most biases among the ROBINS-I domains. We found biases related to non-differential misclassification that were not explicitly covered by ROBINS-I.

Conclusions: Causal structures are helpful to understand biases. By adopting this framework for the interpretation of bias, we show that ROBINS-I covers most biases that may arise in non-randomised studies of interventions. However, further development of the tool should consider bias due to non-differential misclassification.