Updating meta-analyses leads to larger type I errors than publication bias.

OBJECTIVE

To estimate the extent to which the practice of periodically updating meta-analyses causes inflation of the type I error and then to compare the estimate with the inflation caused by publication bias. We also present a simple method to adjust for the inflation associated with updating meta-analyses.

STUDY DESIGN AND SETTING

Simulations were used to estimate the error rates.

RESULTS

In general, updating meta-analyses caused 2- to 5-fold inflation of the type I error rates, which exceeded the inflation caused by publication bias. As a rule of thumb, the results of a meta-analysis are robust up to 5, 10, 15, or 22 updates, if the P-value multiplied by 4, 6, 8, or 10 remains below the desired significance level.

CONCLUSION

Meta-analyses are likely to be updated until a clear conclusion is reached. Therefore, it is important to take the inflation of the error rate into account to interpret the results correctly.