Goldacre Ben, Drysdale Henry, Marston Cicely, Mahtani Kamal R, Dale Aaron, Milosevic Ioan, Slade Eirion, Hartley Philip, Heneghan Carl
Centre for Evidence-Based Medicine, Department of Primary Care Health Sciences, University of Oxford, Radcliffe Observatory Quarter, Woodstock Road, Oxford, OX2 6GG, UK.
Department of Social and Environmental Health Research, London School of Hygiene and Tropical Medicine, Keppel Street, London, WC1E 7HT, UK.
Trials. 2019 Feb 14;20(1):124. doi: 10.1186/s13063-019-3172-3.
Discrepancies between pre-specified and reported outcomes are an important and prevalent source of bias in clinical trials. COMPare (Centre for Evidence-Based Medicine Outcome Monitoring Project) monitored all trials in five leading journals for correct outcome reporting, submitted correction letters on all misreported trials in real time, and then monitored responses from editors and trialists. From the trialists' responses, we aimed to answer two related questions. First, what can trialists' responses to corrections on their own misreported trials tell us about trialists' knowledge of correct outcome reporting? Second, what can a cohort of responses to a standardised correction letter tell us about how researchers respond to systematic critical post-publication peer review?
All correspondence from trialists, published by journals in response to a correction letter from COMPare, was filed and indexed. We analysed the letters qualitatively and identified key themes in researchers' errors about correct outcome reporting, and approaches taken by researchers when their work was criticised.
Trialists frequently expressed views that contradicted the CONSORT (Consolidated Standards of Reporting Trials) guidelines or made inaccurate statements about correct outcome reporting. Common themes were: stating that pre-specification after trial commencement is acceptable; incorrect statements about registries; incorrect statements around the handling of multiple time points; and failure to recognise the need to report changes to pre-specified outcomes in the trial report. We identified additional themes in the approaches taken by researchers when responding to critical correspondence, including the following: ad hominem criticism; arguing that trialists should be trusted, rather than follow guidelines for trial reporting; appealing to the existence of a novel category of outcomes whose results need not necessarily be reported; incorrect statements by researchers about their own paper; and statements undermining transparency infrastructure, such as trial registers.
Researchers commonly make incorrect statements about correct trial reporting. There are recurring themes in researchers' responses when their work is criticised, some of which fall short of the scientific ideal. Research on methodological shortcomings is now common, typically in the form of retrospective cohort studies describing the overall prevalence of a problem. We argue that prospective cohort studies which additionally issue correction letters in real time on each individual flawed study-and then follow-up responses from trialists and journals-are more impactful, more informative for those consuming the studies critiqued, more informative on the causes of shortcomings in research, and a better use of research resources.
预先指定的结果与报告的结果之间存在差异是临床试验中一个重要且普遍存在的偏差来源。COMPare（循证医学结果监测项目中心）监测了五家顶级期刊上的所有试验，以确保结果报告正确，对所有报告有误的试验实时提交了更正信，然后监测编辑和试验者的回复。基于试验者的回复，我们旨在回答两个相关问题。第一，试验者对其自身报告有误的试验的更正回复能让我们了解试验者对正确结果报告的了解程度吗？第二，一组对标准化更正信的回复能让我们了解研究人员如何回应系统性的发表后同行评议批评吗？
期刊发表的试验者的所有回复信件，都是针对COMPare的更正信，这些信件已归档并建立索引。我们对这些信件进行了定性分析，确定了研究人员在正确结果报告方面的错误关键主题，以及研究人员在其工作受到批评时所采取的方法。
试验者经常表达与CONSORT（试验报告统一标准）指南相矛盾的观点，或者对正确结果报告做出不准确的陈述。常见主题包括：声称试验开始后进行预先指定是可以接受的；对注册库的错误陈述；关于多个时间点处理的错误陈述；以及未认识到在试验报告中需要报告对预先指定结果的更改。我们在研究人员回应批评信件时所采取的方法中确定了其他主题，包括以下内容：人身攻击式批评；认为应该信任试验者，而不是遵循试验报告指南；声称存在一类新的结果，其结果不一定需要报告；研究人员对自己论文的错误陈述；以及破坏透明度基础设施（如试验注册库）的陈述。
研究人员在正确的试验报告方面通常会做出错误陈述。研究人员在其工作受到批评时的回复中存在反复出现的主题，其中一些不符合科学理想。目前对方法学缺陷的研究很常见，通常是以回顾性队列研究的形式描述问题的总体发生率。我们认为，前瞻性队列研究更具影响力，对阅读被批评研究的人更有参考价值，能更深入了解研究缺陷的原因，并且能更好地利用研究资源。这种研究形式除了对每个有缺陷的个体研究实时发出更正信外，还会跟踪试验者和期刊的回复。
