In meta-analyses of proportion studies, funnel plots were found to be an inaccurate method of assessing publication bias.

OBJECTIVE

To assess the utility of funnel plots in assessing publication bias (PB) in meta-analyses of proportion studies.

STUDY DESIGN AND SETTING

Meta-analysis simulation study and meta-analysis of published literature reporting peri-operative mortality after abdominal aortic aneurysm (AAA) repair. Data for the simulation study were stochastically generated. A literature search of Medline and Embase was performed to identify studies for inclusion in the published literature meta-analyses.

RESULTS

The simulation study demonstrated that conventionally constructed funnel plots (log odds vs. 1/standard error [1/SE]) for extreme proportional outcomes were asymmetric despite no PB. Alternative funnel plots constructed using study size rather than 1/SE showed no asymmetry for extreme proportional outcomes. When used in meta-analyses of the mortality of AAA repair, these alternative funnel plots highlighted the possibility for conventional funnel plots to demonstrate asymmetry when there was no evidence of PB.

CONCLUSION

Conventional funnel plots used to assess for potential PB in meta-analyses are inaccurate for meta-analyses of proportion studies with low proportion outcomes. Funnel plots of study size against log odds may be a more accurate way of assessing for PB in these studies.