Evaluation of underlying risk as a source of heterogeneity in meta-analyses: a simulation study of Bayesian and frequentist implementations of three models.

Evaluation of important causes of heterogeneity among study results is an important component of any meta-analysis. For factors which can be measured (e.g. population characteristics, indicators of study quality), standard methods such as meta-regression can be used for this evaluation. The underlying risk (i.e. risk of outcome in the control population) can be viewed as a summary of the effects of unmeasured population characteristics so it is a logical candidate for evaluation as a source of heterogeneity. Unfortunately, because of its relationship with the study outcome (odds ratio or relative risk), standard methods should not be used for evaluating underlying risk as a cause of heterogeneity. Three models with different sets of underlying assumptions were evaluated in a simulation study to determine how well they performed in assessing the role of underlying risk as a source of heterogeneity. All models were fit using both Bayesian and frequentist (maximum likelihood random slopes models) estimation procedures and the results compared. Two of the models produced good results (i.e. minimal evidence of bias in parameter estimates), while the third clearly produced biased estimates of some parameters. In general, the Bayesian and frequentist approaches produced similar results. In situations in which the number of studies in a meta-analysis is small ( approximately 10), the maximum likelihood (frequentist) approach was preferable. While the bias induced by heterogeneity associated with underlying risk was generally not large, use of one of the approaches described in this paper will produce better estimates of treatment effect in situations where there is substantial heterogeneity between studies. A model based on the assumption that the number of positive events in each of the treatment and control groups are binomially distributed (Model 1) is the recommended approach.