Department of Medicine, John H. Stroger, Jr. Hospital of Cook County, Chicago, Illinois.
Department of Medicine, Dow University of Health Sciences, Karachi, Pakistan.
JAMA Netw Open. 2020 Apr 1;3(4):e203082. doi: 10.1001/jamanetworkopen.2020.3082.
Multiple analyses in a clinical trial can increase the probability of inaccurately concluding that there is a statistically significant treatment effect. However, to date, it is unknown how many randomized clinical trials (RCTs) perform adjustments for multiple comparisons, the lack of which could lead to erroneous findings.
To assess the prevalence of multiplicity and whether appropriate multiplicity adjustments were performed among cardiovascular RCTs published in 6 medical journals with a high impact factor.
DESIGN, SETTING, AND PARTICIPANTS: In this cross-sectional study, cardiovascular RCTs were selected from all over the world, characterized as North America, Western Europe, multiregional, and rest of the world. Data were collected from past issues of 3 cardiovascular journals (Circulation, European Heart Journal, and Journal of the American College of Cardiology) and 3 general medicine journals (JAMA, The Lancet, and The New England Journal of Medicine) with high impact factors published between August 1, 2015, and July 31, 2018. Supplements and trial protocols of each of the included RCTs were also searched for multiplicity. Data were analyzed December 20 to 27, 2018.
Data from the selected RCTs were extracted and verified independently by 2 researchers using a structured data instrument. In case of disagreement, a third reviewer helped to achieve consensus. An RCT was considered to have multiple treatment groups if it had more than 2 arms; multiple outcomes were defined as having more than 1 primary outcome, and multiple analyses were defined as analysis of the same outcome variable in multiple ways. Multiplicity was examined only for the analysis of the primary end point.
Outcomes of interest were percentages of primary analyses that performed multiplicity adjustment of primary end points.
Of 511 cardiovascular RCTs included in this analysis, 300 (58.7%) had some form of multiplicity; of these 300, only 85 (28.3%) adjusted for multiplicity. Intervention type and funding source had no statistically significant association with the reporting of multiplicity risk adjustment. Trials that assessed mortality vs nonmortality outcomes were more likely to contain a multiplicity risk in their primary analysis (66.3% [177 of 267] vs 50.4% [123 of 244]; P < .001), and larger trials vs smaller trials were less likely to make any adjustments for multiplicity (35.6% [52 of 146] vs 21.4% [33 of 154]; P = .001).
Findings from this study suggest that cardiovascular RCTs published in medical journals with high impact factors demonstrate infrequent adjustments to correct for multiple comparisons in the primary end point. These parameters may be improved by more standardized reporting.
临床试验中的多次分析会增加不准确地得出存在统计学显著治疗效果的概率。然而，迄今为止，尚不清楚有多少随机临床试验（RCT）进行了多次比较的调整，而缺乏这种调整可能会导致错误的发现。
评估心血管 RCT 在 6 种高影响因子医学期刊中发表时的多发性以及是否进行了适当的多发性调整。
设计、设置和参与者：在这项横断面研究中，从世界各地选择了心血管 RCT，分为北美、西欧、多区域和世界其他地区。数据来自过去的 3 种心血管期刊（《循环》、《欧洲心脏杂志》和《美国心脏病学会杂志》）和 3 种普通医学期刊（《美国医学会杂志》、《柳叶刀》和《新英格兰医学杂志》），这些期刊的影响因子均较高，发表时间为 2015 年 8 月 1 日至 2018 年 7 月 31 日。还对每个入选 RCT 的增刊和试验方案进行了多发性搜索。数据分析于 2018 年 12 月 20 日至 27 日进行。
两名研究人员使用结构化数据工具独立提取和验证了所选 RCT 的数据。如果存在分歧，第三名评审员将协助达成共识。如果 RCT 有超过 2 个臂，则认为它有多个治疗组；如果有超过 1 个主要结局，则认为有多个结局，如果对同一结局变量进行了多种分析，则认为存在多次分析。仅对主要终点的主要分析检查了多发性。
感兴趣的结果是对主要终点进行主要分析的多发性调整的百分比。
在本分析中纳入的 511 项心血管 RCT 中，有 300 项（58.7%）存在某种形式的多发性；在这 300 项中，只有 85 项（28.3%）进行了多发性调整。干预类型和资金来源与报告多发性风险调整无统计学显著关联。评估死亡率与非死亡率结局的试验在其主要分析中更有可能存在多发性风险（66.3%[267 例中的 177 例] vs 50.4%[244 例中的 123 例]；P < .001），而较大的试验比较小的试验更不可能进行任何多发性调整（35.6%[146 例中的 52 例] vs 21.4%[154 例中的 33 例]；P = .001）。
本研究结果表明，在高影响因子医学期刊中发表的心血管 RCT 很少对主要终点的多次比较进行调整以纠正这种情况。通过更标准化的报告，这些参数可能会得到改善。
