Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK.
Applied Health Research Centre, Li Ka Shing Knowledge Institute of St Michael's Hospital, Toronto, Ontario, Canada.
Int J Epidemiol. 2020 Jun 1;49(3):968-978. doi: 10.1093/ije/dyaa026.
It is unclear how multiple treatment comparisons are managed in the analysis of multi-arm trials, particularly related to reducing type I (false positive) and type II errors (false negative).
We conducted a cohort study of clinical-trial protocols that were approved by research ethics committees in the UK, Switzerland, Germany and Canada in 2012. We examined the use of multiple-testing procedures to control the overall type I error rate. We created a decision tool to determine the need for multiple-testing procedures. We compared the result of the decision tool to the analysis plan in the protocol. We also compared the pre-specified analysis plans in trial protocols to their publications.
Sixty-four protocols for multi-arm trials were identified, of which 50 involved multiple testing. Nine of 50 trials (18%) used a single-step multiple-testing procedures such as a Bonferroni correction and 17 (38%) used an ordered sequence of primary comparisons to control the overall type I error. Based on our decision tool, 45 of 50 protocols (90%) required use of a multiple-testing procedure but only 28 of the 45 (62%) accounted for multiplicity in their analysis or provided a rationale if no multiple-testing procedure was used. We identified 32 protocol-publication pairs, of which 8 planned a global-comparison test and 20 planned a multiple-testing procedure in their trial protocol. However, four of these eight trials (50%) did not use the global-comparison test. Likewise, 3 of the 20 trials (15%) did not perform the multiple-testing procedure in the publication. The sample size of our study was small and we did not have access to statistical-analysis plans for the included trials in our study.
Strategies to reduce type I and type II errors are inconsistently employed in multi-arm trials. Important analytical differences exist between planned analyses in clinical-trial protocols and subsequent publications, which may suggest selective reporting of analyses.
目前尚不清楚如何在多臂试验的分析中管理多次治疗比较，特别是与降低 I 类错误（假阳性）和 II 类错误（假阴性）有关。
我们对 2012 年在英国、瑞士、德国和加拿大获得研究伦理委员会批准的临床试验方案进行了队列研究。我们考察了使用多重检验程序来控制总体 I 类错误率的情况。我们创建了一个决策工具来确定是否需要使用多重检验程序。我们将决策工具的结果与方案中的分析计划进行了比较。我们还比较了试验方案中预先指定的分析计划与其出版物。
确定了 64 项多臂试验方案，其中 50 项涉及多次检验。在这 50 项试验中，有 9 项（18%）使用了单一步骤的多重检验程序，如 Bonferroni 校正，有 17 项（38%）使用了按顺序排列的主要比较来控制总体 I 类错误。根据我们的决策工具，50 项方案中的 45 项（90%）需要使用多重检验程序，但其中只有 28 项（62%）在分析中考虑了多重性，或者如果没有使用多重检验程序，则提供了理由。我们确定了 32 对方案-出版物，其中 8 项计划在试验方案中进行全局比较检验，20 项计划进行多重检验程序。然而，这 8 项试验中有 4 项（50%）没有使用全局比较检验。同样，在 20 项试验中有 3 项（15%）没有在出版物中执行多重检验程序。我们研究的样本量较小，并且我们无法获得纳入研究的试验的统计分析计划。
在多臂试验中，降低 I 类和 II 类错误的策略使用不一致。临床试验方案中的计划分析与随后的出版物之间存在重要的分析差异，这可能表明分析存在选择性报告。
