Statistical methods can be improved within Cochrane pregnancy and childbirth reviews.

OBJECTIVES

To assess statistical methods within systematic reviews of the Cochrane Pregnancy and Childbirth Group (CPCG).

STUDY DESIGN AND SETTING

We extracted details about statistical methods within 75 reviews containing at least 10 studies.

RESULTS

The median number of forest plots per review was 52 (min=5; max=409). Seven of the 75 reviews assessed publication bias or explained why not. Forty-four of the 75 reviews performed random-effects meta-analyses; just 1 of these justified the approach clinically and none interpreted its pooled result correctly. Of 31 reviews not using random-effects, 26 assumed a fixed-effect given potentially moderate or large heterogeneity (I(2)>25%). In their Methods section, 25 (33%) of the 75 reviews said I(2) was used to decide between fixed-effect and random-effects; however, in 12 of these (48%) reviews, this was not carried out in their Results section. Of 72 reviews with moderate or large heterogeneity, 47 (65%) did not explore the causes of heterogeneity or justify why not.

CONCLUSION

Within CPCG reviews, publication bias is rarely addressed; heterogeneity is often not appropriately considered, and random-effects analyses are incorrectly interpreted. How these shortcomings impact existing review conclusions needs further investigation, but regardless of this, we recomment the Cochrane Collaboration increase "hands-on" statistical support.