Université de Paris, CRESS, Inserm, INRA, F75004, Paris, France.
Centre d'Épidémiologie Clinique, Hôpital Hôtel Dieu, Assistance Publique des Hôpitaux de Paris, Paris, France.
BMC Med. 2019 Nov 19;17(1):205. doi: 10.1186/s12916-019-1436-0.
BACKGROUND: The peer review process has been questioned as it may fail to allow the publication of high-quality articles. This study aimed to evaluate the accuracy in identifying inadequate reporting in RCT reports by early career researchers (ECRs) using an online CONSORT-based peer-review tool (COBPeer) versus the usual peer-review process. METHODS: We performed a cross-sectional diagnostic study of 119 manuscripts, from BMC series medical journals, BMJ, BMJ Open, and Annals of Emergency Medicine reporting the results of two-arm parallel-group RCTs. One hundred and nineteen ECRs who had never reviewed an RCT manuscript were recruited from December 2017 to January 2018. Each ECR assessed one manuscript. To assess accuracy in identifying inadequate reporting, we used two tests: (1) ECRs assessing a manuscript using the COBPeer tool (after completing an online training module) and (2) the usual peer-review process. The reference standard was the assessment of the manuscript by two systematic reviewers. Inadequate reporting was defined as incomplete reporting or a switch in primary outcome and considered nine domains: the eight most important CONSORT domains and a switch in primary outcome(s). The primary outcome was the mean number of domains accurately classified (scale from 0 to 9). RESULTS: The mean (SD) number of domains (0 to 9) accurately classified per manuscript was 6.39 (1.49) for ECRs using COBPeer versus 5.03 (1.84) for the journal's usual peer-review process, with a mean difference [95% CI] of 1.36 [0.88-1.84] (p < 0.001). Concerning secondary outcomes, the sensitivity of ECRs using COBPeer versus the usual peer-review process in detecting incompletely reported CONSORT items was 86% [95% CI 82-89] versus 20% [16-24] and in identifying a switch in primary outcome 61% [44-77] versus 11% [3-26]. The specificity of ECRs using COBPeer versus the usual process to detect incompletely reported CONSORT domains was 61% [57-65] versus 77% [74-81] and to identify a switch in primary outcome 77% [67-86] versus 98% [92-100]. CONCLUSIONS: Trained ECRs using the COBPeer tool were more likely to detect inadequate reporting in RCTs than the usual peer review processes used by journals. Implementing a two-step peer-review process could help improve the quality of reporting. TRIAL REGISTRATION: Clinical.Trials.gov NCT03119376 (Registered April, 18, 2017).
背景：同行评议过程受到质疑，因为它可能无法发表高质量的文章。本研究旨在使用基于 CONSORT 的在线同行评议工具（COBPeer）评估早期职业研究人员（ECR）识别 RCT 报告中报告不足的准确性，与常规同行评议过程相比。
方法：我们对来自 BMC 系列医学期刊、BMJ、BMJ Open 和《急诊医学年鉴》的 119 篇报告两臂平行 RCT 结果的手稿进行了横断面诊断研究。从 2017 年 12 月至 2018 年 1 月，招募了从未评审过 RCT 手稿的 119 名 ECR。每位 ECR 评估一篇手稿。为了评估识别报告不足的准确性，我们使用了两种测试：（1）ECR 使用 COBPeer 工具（完成在线培训模块后）评估手稿，（2）常规同行评审过程。参考标准是由两名系统审查员评估手稿。报告不足定义为不完整报告或主要结局的改变，并考虑了九个领域：八个最重要的 CONSORT 领域和主要结局的改变。主要结局是准确分类的领域数量（范围为 0 到 9）。
结果：ECR 使用 COBPeer 评估的手稿每篇平均（SD）正确分类的领域数量为 6.39（1.49），而期刊常规同行评审过程为 5.03（1.84），平均差异[95%CI]为 1.36 [0.88-1.84]（p < 0.001）。关于次要结局，与常规同行评审过程相比，ECR 使用 COBPeer 检测不完整报告的 CONSORT 项目的灵敏度为 86%[95%CI 82-89]对 20%[16-24]和识别主要结局的改变为 61%[44-77]对 11%[3-26]。与常规过程相比，ECR 使用 COBPeer 检测不完整报告的 CONSORT 领域的特异性为 61%[57-65]对 77%[74-81]，识别主要结局的改变为 77%[67-86]对 98%[92-100]。
结论：经过培训的 ECR 使用 COBPeer 工具比期刊使用的常规同行评审过程更有可能检测到 RCT 中的报告不足。实施两步同行评审过程可以帮助提高报告质量。
试验注册：Clinical.Trials.gov NCT03119376（2017 年 4 月 18 日注册）。
Cochrane Database Syst Rev. 2022-2-1
Cochrane Database Syst Rev. 2023-11-28
PLoS Med. 2025-4-28
PLoS Med. 2025-4-14
Science. 2013-10-4
PLoS One. 2013-4-10
Zotero 插件
