Department of Statistics, Florida State University, Tallahassee, FL, USA.
Department of Mathematics, College of Science and Arts, Najran University, Najran, Saudi Arabia.
BMC Med. 2023 Mar 29;21(1):112. doi: 10.1186/s12916-023-02823-9.
Studies included in a meta-analysis are often heterogeneous. The traditional random-effects models assume their true effects to follow a normal distribution, while it is unclear if this critical assumption is practical. Violations of this between-study normality assumption could lead to problematic meta-analytical conclusions. We aimed to empirically examine if this assumption is valid in published meta-analyses.
In this cross-sectional study, we collected meta-analyses available in the Cochrane Library with at least 10 studies and with between-study variance estimates > 0. For each extracted meta-analysis, we performed the Shapiro-Wilk (SW) test to quantitatively assess the between-study normality assumption. For binary outcomes, we assessed between-study normality for odds ratios (ORs), relative risks (RRs), and risk differences (RDs). Subgroup analyses based on sample sizes and event rates were used to rule out the potential confounders. In addition, we obtained the quantile-quantile (Q-Q) plot of study-specific standardized residuals for visually assessing between-study normality.
Based on 4234 eligible meta-analyses with binary outcomes and 3433 with non-binary outcomes, the proportion of meta-analyses that had statistically significant non-normality varied from 15.1 to 26.2%. RDs and non-binary outcomes led to more frequent non-normality issues than ORs and RRs. For binary outcomes, the between-study non-normality was more frequently found in meta-analyses with larger sample sizes and event rates away from 0 and 100%. The agreements of assessing the normality between two independent researchers based on Q-Q plots were fair or moderate.
The between-study normality assumption is commonly violated in Cochrane meta-analyses. This assumption should be routinely assessed when performing a meta-analysis. When it may not hold, alternative meta-analysis methods that do not make this assumption should be considered.
荟萃分析中包含的研究通常是异质的。传统的随机效应模型假设其真实效应遵循正态分布，但尚不清楚这一关键假设是否实际。违反这一研究间正态性假设可能导致有问题的荟萃分析结论。我们旨在实证检验这一假设在已发表的荟萃分析中是否成立。
在这项横断面研究中，我们收集了 Cochrane 图书馆中至少包含 10 项研究且研究间方差估计值>0 的荟萃分析。对于每个提取的荟萃分析，我们进行 Shapiro-Wilk（SW）检验，以定量评估研究间正态性假设。对于二分类结局，我们评估了比值比（ORs）、相对风险（RRs）和风险差（RDs）的研究间正态性。基于样本量和事件率的亚组分析用于排除潜在的混杂因素。此外，我们获得了研究特异性标准化残差的分位数-分位数（Q-Q）图，以直观评估研究间正态性。
基于 4234 项有二分类结局的合格荟萃分析和 3433 项无二分类结局的荟萃分析，具有统计学显著非正态性的荟萃分析比例从 15.1%到 26.2%不等。RDs 和无二分类结局比 ORs 和 RRs 更容易出现非正态性问题。对于二分类结局，研究间非正态性更常见于样本量较大且事件率远离 0 和 100%的荟萃分析。基于 Q-Q 图评估两位独立研究者的正态性的一致性为中等或适度。
Cochrane 荟萃分析中普遍违反了研究间正态性假设。在进行荟萃分析时，应常规评估这一假设。当它可能不成立时，应考虑不做此假设的替代荟萃分析方法。
