Esterhuizen Tonya Marianne, Mbuagbaw Lawrence, Rehman Nadia, Yanwou Nathan, Swaby Devron J, Kittle Esme, Licht Johann-Christoph, Thabane Lehana
Division of Epidemiology and Biostatistics, Department of Global Health, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, Western Cape, South Africa
Division of Epidemiology and Biostatistics, Department of Global Health, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, Western Cape, South Africa.
BMJ Open. 2025 Aug 25;15(8):e100411. doi: 10.1136/bmjopen-2025-100411.
We estimated the extent of the disparity between statistical significance and clinical importance in published randomised controlled trials (RCTs), and explored factors associated with this disparity.
A methodological study of trials published between 2018 and 2022 and indexed in PubMed was conducted. Primary reports of two-arm, phase three, superiority trials of human health interventions were included. Pharmacokinetic studies and pilot trials were excluded. The relationship between the specified delta value or minimum clinically important difference (as specified in the sample size calculation) and the effect size determined the clinical importance of the trial results. Studies where the clinical importance was at least possible, with no statistical significance, were classified as SS-CI+ disparity, and studies which were definitely not clinically important but statistically significant were classified as SS+CI- disparity. Factors associated with each type of disparity were explored at the study level using multinomial logistic regression.
500 trials were included. In 38.4% (n=192) of these, information was not available to classify clinical importance. Overall disparity was found in 63 of the remaining 308 studies, 20.5% (95% confidence interval (CI) 16.2% to 25.5%). SS+CI- disparity was 10.3% (15/145) (95% CI 6.1% to 16.8%) and SS-CI+ disparity was 29.5% (48/163) (95% CI 22.7% to 37.2%).Studies testing complementary or alternative medicines relative to drug trials were positively associated with SS+CI- disparity. Low journal impact factor, small sample size, unfunded or grant funding and failure to mention allocation concealment were positively associated with SS-CI+disparity.
In up to 20% of RCTs, there may be a disparity between statistical significance and clinical importance. Clinical importance of results should be taken into account in the interpretation of trial results, and trials should adhere stringently to reporting guidelines.
我们估计了已发表的随机对照试验（RCT）中统计学显著性与临床重要性之间的差异程度，并探讨了与这种差异相关的因素。
对2018年至2022年间发表并被PubMed收录的试验进行了一项方法学研究。纳入了人类健康干预的双臂、三期、优效性试验的主要报告。排除了药代动力学研究和试点试验。指定的δ值或最小临床重要差异（如样本量计算中所规定）与效应大小之间的关系决定了试验结果的临床重要性。临床重要性至少有可能但无统计学显著性的研究被归类为SS-CI+差异，而肯定无临床重要性但有统计学显著性的研究被归类为SS+CI-差异。使用多项逻辑回归在研究层面探讨与每种差异类型相关的因素。
纳入了500项试验。其中38.4%（n = 192）的试验没有可用于分类临床重要性的信息。在其余308项研究中的63项中发现了总体差异，占20.5%（95%置信区间（CI）16.2%至25.5%）。SS+CI-差异为10.3%（15/145）（95% CI 6.1%至16.8%），SS-CI+差异为29.5%（48/163）（95% CI 22.7%至37.2%）。与药物试验相比，测试补充或替代药物的研究与SS+CI-差异呈正相关。低期刊影响因子、小样本量、无资金资助或赠款资助以及未提及分配隐藏与SS-CI+差异呈正相关。
在高达20%的RCT中，统计学显著性与临床重要性之间可能存在差异。在解释试验结果时应考虑结果的临床重要性，并且试验应严格遵守报告指南。
