Santos Joseph Alvin Ramos, Riggi Emilia, Di Tanna Gian Luca
Department of Business Economics, Health and Social Care (DEASS), University of Applied Sciences and Arts of Southern Switzerland (SUPSI), Manno, Ticino, Switzerland.
The George Institute for Global Health, Sydney, Australia.
BMC Med Res Methodol. 2025 May 15;25(1):133. doi: 10.1186/s12874-025-02586-2.
Consolidation of treatment effects from randomized controlled trials (RCT) is considered one of the highest forms of evidence in research. Cluster randomized trials (CRT) are increasingly used in the assessment of the effectiveness of interventions when individual-level randomization is impractical. In meta-analyses, CRTs that address the same clinical question as RCTs can be pooled in the same analysis; however, they need to be analyzed with appropriate statistical methods. This study examined the extent to which meta-analysis results are influenced by the inclusion of incorrectly analyzed CRTs through a series of simulations.
RCT and CRT datasets were generated with a continuous treatment effect of zero, two trial arms, and equal number of participants. CRT datasets were generated with varying number of clusters (10, 20 or 40), observations per cluster (10, 30 or 50), total variance (1, 5 or 10) and ICC (0.05, 0.10 or 0.20). Each simulated CRT dataset (n = 1000 for each scenario) was analyzed using standard linear regression and mixed-effects regression with clusters treated as random effects to represent the incorrectly and correctly analyzed CRTs. Meta-analytic datasets were created by varying the total number of studies (4, 8 or 12), number of CRTs out of the total number of studies (single, half or all), and the number of correctly analyzed CRTs (none, half or all). Model performance was summarized from 1000 random-effects meta-analyses for each scenario.
The percentage of statistically significant results (at p < 0.05) was consistently lower when all CRTs were correctly analyzed. The alpha threshold (5%) was exceeded in 6 (2.47%) of 243 scenarios when all CRTs were correctly analyzed, compared to 177 (72.84%) and 195 (80.25%) scenarios when half or none of the CRTs were correctly analyzed, respectively. Coverage probabilities and model-based SEs were higher when all CRTs were correctly analyzed, while the estimated effect sizes and bias averaged across iterations showed no differences regardless of the number of correctly analyzed CRTs.
Ignoring clustering in CRTs lead to inflated false-positive conclusions about the efficacy of treatments, highlighting the need for caution and proper analytical methods when incorporating CRTs into meta-analyses.
随机对照试验（RCT）治疗效果的合并被认为是研究中最高形式的证据之一。当个体水平的随机化不可行时，整群随机试验（CRT）越来越多地用于评估干预措施的有效性。在荟萃分析中，与RCT解决相同临床问题的CRT可以在同一分析中进行汇总；然而，它们需要用适当的统计方法进行分析。本研究通过一系列模拟检验了纳入分析错误的CRT对荟萃分析结果的影响程度。
生成具有零连续治疗效果、两个试验组和相等参与者数量的RCT和CRT数据集。生成的CRT数据集具有不同数量的聚类（10、20或40）、每个聚类的观察值（10、30或50）、总方差（1、5或10）和组内相关系数（ICC，0.05、0.10或0.20）。每个模拟的CRT数据集（每个场景n = 1000）使用标准线性回归和将聚类视为随机效应的混合效应回归进行分析，以分别代表分析错误和正确的CRT。通过改变研究总数（4、8或12）、研究总数中CRT的数量（单个、一半或全部）以及正确分析的CRT数量（无、一半或全部）来创建荟萃分析数据集。对每个场景的1000次随机效应荟萃分析总结模型性能。
当所有CRT都被正确分析时，具有统计学显著性结果（p < 0.05）的百分比始终较低。当所有CRT都被正确分析时，243个场景中有6个（2.47%）超过了α阈值（5%），而当一半或没有CRT被正确分析时，分别为177个（72.84%）和195个（80.25%）场景。当所有CRT都被正确分析时，覆盖概率和基于模型的标准误更高，而无论正确分析的CRT数量如何，迭代过程中平均的估计效应大小和偏差均无差异。
忽略CRT中的聚类会导致关于治疗效果的假阳性结论膨胀，突出了在将CRT纳入荟萃分析时需要谨慎并采用适当分析方法的必要性。
