Department of Anaesthesia, Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, UK.
School of Medicine and Population Health, University of Sheffield, Sheffield, UK.
Anaesthesia. 2024 Dec;79(12):1309-1316. doi: 10.1111/anae.16411. Epub 2024 Aug 15.
There is some evidence for systematic biases and failures of research integrity in the anaesthesia literature. However, the features of problematic trials and effect of editorial selection on these issues have not been well quantified.
We analysed 209 randomised controlled trials submitted to Anaesthesia between 8 March 2019 and 31 March 2020. We evaluated the submitted manuscript, registry data and the results of investigations into the integrity of the trial undertaken at the time of submission. Trials were labelled 'concerning' if failures of research integrity were found, and 'problematic' if identified issues would have warranted retraction if they had been found after publication. We investigated how 'problematic' trials were detected, the distribution of p values and the risk of outcome reporting bias and p-hacking. We also investigated whether there were any factors that differed in problematic trials.
We found that false data was the most common reason for a trial to be labelled as 'concerning', which occurred in 51/62 (82%) cases. We also found that while 195/209 (93%) trials were preregistered, we found adequate registration for only 166/209 (79%) primary outcomes, 100/209 (48%) secondary outcomes and 11/209 (5%) analysis plans. We also found evidence for a step decrease in the frequency of p values > 0.05 compared with p values < 0.05. 'Problematic' trials were all single-centre and appeared to have fewer authors (incident risk ratio (95%CI) 0.8 (0.7-0.9)), but could not otherwise be distinguished reliably from other trials.
Identification of 'problematic' trials is frequently dependent on individual patient data, which is often unavailable after publication. Additionally, there is evidence of a risk of outcome reporting bias and p-hacking in submitted trials. Implementation of alternative research and editorial practices could reduce the risk of bias and make identification of problematic trials easier.
有证据表明，麻醉学文献中存在系统偏差和研究诚信失败。然而，有问题的试验的特征以及编辑选择对这些问题的影响尚未得到很好的量化。
我们分析了 2019 年 3 月 8 日至 2020 年 3 月 31 日期间提交给《麻醉学》的 209 项随机对照试验。我们评估了提交的手稿、注册数据以及在提交时对试验完整性进行调查的结果。如果发现研究诚信失败，试验将被标记为“有关”，如果发现有问题的问题，如果在发表后发现这些问题，则将被标记为“有问题”。我们调查了“有问题”的试验是如何被发现的，p 值的分布，以及结果报告偏倚和 p 值操纵的风险。我们还调查了是否有任何因素在有问题的试验中有所不同。
我们发现，虚假数据是将试验标记为“有关”的最常见原因，在 62 个试验中有 51 个（82%）发生这种情况。我们还发现，虽然 209 个试验中有 195 个（93%）进行了预先登记，但我们仅对 209 个（79%）主要结局、209 个（48%）次要结局和 209 个（5%）分析计划进行了充分登记。我们还发现，与 p 值 < 0.05 相比，p 值 > 0.05 的频率呈阶梯式下降。“有问题”的试验都是单中心的，似乎作者较少（发病风险比（95%CI）0.8（0.7-0.9）），但其他方面与其他试验无法可靠区分。
“有问题”试验的识别通常依赖于个体患者数据，而这些数据在发表后往往无法获得。此外，提交的试验中存在结果报告偏倚和 p 值操纵的风险。实施替代的研究和编辑实践可以降低偏差风险，并使识别有问题的试验变得更容易。
