Division of Infectious Diseases, Queen's University, Kingston, ON, Canada.
Health Research Methodology Program, McMaster University, Hamilton, ON, Canada.
Trials. 2021 Oct 16;22(1):708. doi: 10.1186/s13063-021-05686-8.
Numerous statistical methods can be used to calculate the confidence interval (CI) of risk differences. There is consensus in previous literature that the Wald method should be discouraged. We compared five statistical methods for estimating the CI of risk difference in terms of CI width and study conclusion in antibiotic non-inferiority trials.
In a secondary analysis of a systematic review, we included non-inferiority trials that compared different antibiotic regimens, reported risk differences for the primary outcome, and described the number of successes and/or failures as well as patients in each arm. For each study, we re-calculated the risk difference CI using the Wald, Agresti-Caffo, Newcombe, Miettinen-Nurminen, and skewness-corrected asymptotic score (SCAS) methods. The CIs by different statistical methods were compared in terms of CI width and conclusion on non-inferiority. A wider CI was considered to be more conservative.
The analysis included 224 comparisons from 213 studies. The statistical method used to calculate CI was not reported in 134 (59.8%) cases. The median (interquartile range IQR) for CI width by Wald, Agresti-Caffo, Newcombe, Miettinen-Nurminen, and SCAS methods was 13.0% (10.8%, 17.4%), 13.3% (10.9%, 18.5%), 13.6% (11.1%, 18.9%), 13.6% (11.1% and 19.0%), and 13.4% (11.1%, 18.9%), respectively. In 216 comparisons that reported a non-inferiority margin, the conclusion on non-inferiority was the same across the five statistical methods in 211 (97.7%) cases. The differences in CI width were more in trials with a sample size of 100 or less in each group and treatment success rate above 90%. Of the 18 trials in this subgroup with a specified non-inferiority margin, non-inferiority was shown in 17 (94.4%), 16 (88.9%), 14 (77.8%), 14 (77.8%), and 15 (83.3%) cases based on CI by Wald, Agresti-Caffo, Newcombe, Miettinen-Nurminen, and SCAS methods, respectively.
The statistical method used to calculate CI was not reported in the majority of antibiotic non-inferiority trials. Different statistical methods for CI resulted in different conclusions on non-inferiority in 2.3% cases. The differences in CI widths were highest in trials with a sample size of 100 or less in each group and a treatment success rate above 90%.
PROSPERO CRD42020165040 . April 28, 2020.
有许多统计学方法可用于计算风险差异的置信区间 (CI)。先前的文献中有共识认为应避免 Wald 法。我们比较了五种在抗生素非劣效性试验中估计风险差异 CI 的统计方法，以 CI 宽度和研究结论为指标。
在一项系统评价的二次分析中，我们纳入了比较不同抗生素方案、报告主要结局风险差异且描述了每组成功和/或失败例数及患者数的非劣效性试验。对于每个研究，我们使用 Wald、Agresti-Caffo、Newcombe、Miettinen-Nurminen 和偏度校正渐近得分 (SCAS) 方法重新计算风险差异 CI。不同统计方法的 CI 以 CI 宽度和非劣效性结论进行比较。CI 越宽则被认为越保守。
分析纳入了来自 213 项研究的 224 项比较。134 项（59.8%）研究未报告计算 CI 所使用的统计学方法。Wald、Agresti-Caffo、Newcombe、Miettinen-Nurminen 和 SCAS 方法的 CI 宽度中位数（四分位距 IQR）分别为 13.0%（10.8%，17.4%）、13.3%（10.9%，18.5%）、13.6%（11.1%，18.9%）、13.6%（11.1%和 19.0%）和 13.4%（11.1%，18.9%）。在 216 项报告了非劣效性边界的比较中，5 种统计方法在 211 项（97.7%）研究中得出了相同的非劣效性结论。在每组样本量为 100 或更少和治疗成功率高于 90%的试验中，CI 宽度的差异更大。在这个亚组的 18 项指定了非劣效性边界的试验中，基于 Wald、Agresti-Caffo、Newcombe、Miettinen-Nurminen 和 SCAS 方法的 CI，非劣效性在 17 项（94.4%）、16 项（88.9%）、14 项（77.8%）、14 项（77.8%）和 15 项（83.3%）中得到了证实。
在大多数抗生素非劣效性试验中，未报告计算 CI 所使用的统计学方法。在 2.3%的情况下，不同的 CI 统计方法得出了不同的非劣效性结论。在每组样本量为 100 或更少和治疗成功率高于 90%的试验中，CI 宽度的差异最大。
PROSPERO CRD42020165040。2020 年 4 月 28 日。
