Accuracy and Precision of an Effect Size and Its Variance From a Multilevel Model for Cluster Randomized Trials: A Simulation Study.

This article investigates an effect size (MLM ES) and its variance for cluster randomized trials based on parameter estimates from multilevel modeling analysis. Accuracy and precision of MLM ES were evaluated using Monte Carlo simulation methods and compared with the performance of an effect size, computed from summary statistics, proposed by Hedges (2007; Hedges' dB ). Simulation results indicated that MLM ES had acceptable accuracy in all conditions, also demonstrating efficiency and consistency. With small sample sizes, MLM ES did not suffer from the same negative bias as Hedges' dB due to overestimation of between-cluster variance. With large sample sizes, MLM ES and Hedges' dB were comparable for accuracy and efficiency. Both MLM ES and Hedges' dB showed considerable bias in some conditions when cluster sizes were unequal. An illustrative example using real data was provided.