Division of Epidemiology and Biostatistics, Stellenbosch University, Stellenbosch, South Africa
Division of Epidemiology and Biostatistics, Stellenbosch University, Stellenbosch, South Africa.
BMJ Open. 2024 Jul 25;14(7):e084375. doi: 10.1136/bmjopen-2024-084375.
The commonly used frequentist paradigm of null hypothesis statistics testing with its reliance on the p-value and the corresponding notion of 'statistical significance' has been under ongoing criticism. Misinterpretation and misuse of the p-value have contributed to publication bias, unreliable studies, frequent false positives, fraud and mistrust in results of scientific studies. While p-values themselves are still useful, part of the problem may be the confusion between statistical and clinical significance. In randomised controlled trials of health interventions, this confusion could lead to erroneous conclusions about treatment efficacy, research waste and compromised patient outcomes. The extent to which clinical and statistical significance of published randomised clinical trials do not match is not known. This is a protocol for a methodological study to understand the extent of the problem of disparities between statistical and clinical significance in published clinical trials, and to identify and assess the factors associated with discrepant results in these studies.
A methodological survey of published randomised controlled trials is planned. Trials published between 2018 and 2022 and their protocols will be searched and screened for inclusion, with a planned sample size of 500 studies. The reported minimum clinically important difference, the study effect size and confidence intervals will be used to assess clinical importance of trial results. Comparison of statistical significance and clinical importance of the trial results will be used to determine disparity. Data will be analysed to estimate the outcomes, and factors associated with disparate study results will be assessed using logistic regression analysis.
Ethical approval for the study has been granted by Stellenbosch University's Health Research Ethics Committee. This is part of a larger study towards a PhD in Biostatistics and will be disseminated as a thesis, conference abstract and peer-reviewed manuscript.
常用的假设检验的频率派范式，以及对 p 值的依赖和相应的“统计学意义”概念，一直受到持续的批评。对 p 值的误解和滥用导致了发表偏倚、不可靠的研究、频繁的假阳性、欺诈以及对科学研究结果的不信任。虽然 p 值本身仍然有用，但部分问题可能是统计意义和临床意义之间的混淆。在健康干预的随机对照试验中，这种混淆可能导致对治疗效果的错误结论、研究浪费和患者结果受损。发表的随机临床试验的临床和统计学意义之间不匹配的程度尚不清楚。这是一项方法学研究的方案，旨在了解发表的临床试验中统计意义和临床意义之间差异的程度，并确定和评估这些研究中不一致结果的相关因素。
计划对已发表的随机对照试验进行方法学调查。将搜索并筛选 2018 年至 2022 年期间发表的试验及其方案，计划样本量为 500 项研究。将使用报告的最小临床重要差异、研究效应量和置信区间来评估试验结果的临床重要性。比较试验结果的统计学意义和临床重要性，以确定差异。将对数据进行分析以估计结果，并使用逻辑回归分析评估与不一致研究结果相关的因素。
该研究已获得斯泰伦博斯大学健康研究伦理委员会的伦理批准。这是一项更大的生物统计学博士研究的一部分，将作为论文、会议摘要和同行评审的手稿进行传播。
