Background: Randomised trials vary in methodological quality, and flaws in trial conduct can lead to biased estimation of the intervention effect. Recently, two methods for adjustment of within-trial biases in meta-analysis have been proposed. The first uses empirical evidence on the magnitude of biases observed in a large collection of meta-analyses; the second uses expert opinion informed by detailed assessment of the potential biases affecting each trial.

Objectives: Our aim is to integrate two existing approaches to bias adjustment in order to gain the advantages of both.

Methods: Three different methods for combining empirical evidence on bias and detailed study assessment were considered. Empirical bias distributions for trials with different combinations of risk-of-bias judgements were derived from a hierarchical model fitted to 64 meta-analyses from Cochrane reviews. Opinion-based bias distributions were averaged across four experts who read summary information on each trial in a new meta-analysis, and independently gave their opinions on bias. In the first combined method, empirical evidence and opinion were formally combined in a Bayesian analysis. In two alternative methods, experts were asked to give their opinion based on summary trial information and the empirical bias distribution, either numerically or by selecting areas of the distribution. The methods were compared through application to example meta-analyses.

Results: Numerical results obtained from the three different integrated approaches to bias adjustment were similar. In an example meta-analysis, bias adjustment based on empirical evidence and opinion caused the intervention log odds ratio to shift towards the null by 21%, and between-trial variance reduced substantially by 28%.

Conclusions: Adjustment for biases is useful in meta-analyses synthesising all available evidence. We recommend an integrated approach to bias adjustment, informed by both available empirical evidence and elicited opinion. We discuss the advantages and disadvantages of different approaches to combining evidence on bias with opinion.