Analysis of a cluster randomized trial with binary outcome data using a multi-level model.

The use of multi-level logistic regression models was explored for the analysis of data from a cluster randomized trial investigating whether a training programme for general practitioners' reception staff could improve women's attendance at breast screening. Twenty-six general practices were randomized with women nested within them, requiring a two-level model which allowed for between-practice variability. Comparisons were made with fixed effect (FE) and random effects (RE) cluster summary statistic methods, ordinary logistic regression and a marginal model based on generalized estimating equations with robust variance estimates. An FE summary statistic method and ordinary logistic regression considerably understated the variance of the intervention effect, thus overstating its statistical significance. The marginal model produced a higher statistical significance for the intervention effect compared to that obtained from the RE summary statistic method and the multi-level model. Because there was only a moderate number of practices and these had unbalanced cluster sizes, reliable asymptotic properties for the robust standard errors used in the marginal model may not have been achieved. While the RE summary statistic method cannot handle multiple covariates easily, marginal and multi-level models can do so. In contrast to multi-level models however, marginal models do not provide direct estimates of variance components, but treat these as nuisance parameters. Estimates of the variance components were of particular interest in this example. Additionally, parametric bootstrap methods within the multi-level model framework provide confidence intervals for these variance components, as well as a confidence interval for the effect of intervention which allows for the imprecision in the estimated variance components. The assumption of normality of the random effects can be checked, and the models extended to investigate multiple sources of variability.