作者推荐的审稿人与编辑选择的审稿人一样优秀吗？一项评分者盲法回顾性研究的结果。

Are reviewers suggested by authors as good as those chosen by editors? Results of a rater-blinded, retrospective study.

作者信息

Wager Elizabeth, Parkin Emma C, Tamber Pritpal S

机构信息

Sideview, Princes Risborough, HP27 9DE, UK.

出版信息

BMC Med. 2006 May 30;4:13. doi: 10.1186/1741-7015-4-13.

DOI:10.1186/1741-7015-4-13

PMID:16734897

原文链接:https://pmc.ncbi.nlm.nih.gov/articles/PMC1482713/

Abstract

BACKGROUND

BioMed Central (BMC) requires authors to suggest four reviewers when making a submission. Editors searching for reviewers use these suggestions as a source. The review process of the medical journals in the BMC series is open--authors and reviewers know each other's identity--although reviewers can make confidential comments to the editor. Reviews are published alongside accepted articles so readers may see the reviewers' names and recommendations. Our objective was to compare the performance of author-nominated reviewers (ANR) with that of editor-chosen reviewers (ECR) in terms of review quality and recommendations about submissions in an online-only medical journal.

METHODS

Pairs of reviews from 100 consecutive submissions to medical journals in the BMC series (with one author-nominated and one editor-chosen reviewer and a final decision) were assessed by two raters, blinded to reviewer type, using a validated review quality instrument (RQI) which rates 7 items on 5-point Likert scales. The raters discussed their ratings after the first 20 pairs (keeping reviewer type masked) and resolved major discrepancies in scoring and interpretation to improve inter-rater reliability. Reviewers' recommendations were also compared.

RESULTS

Reviewer source had no impact on review quality (mean RQI score (+/- SD) 2.24 +/- 0.55 for ANR, 2.34 +/- 0.54 for ECR) or tone (mean scores on additional question 2.72 ANR vs 2.82 ECR) (maximum score = 5 in both cases). However author-nominated reviewers were significantly more likely to recommend acceptance (47 vs 35) and less likely to recommend rejection (10 vs 23) than editor-chosen reviewers after initial review (p < 0.001). However, by the final review stage (i.e. after authors had responded to reviewer comments) ANR and ECR recommendations were similar (65 vs 66 accept, 10 vs 14 reject, p = 0.47). The number of reviewers unable to decide about acceptance was similar in both groups at both review stages.

CONCLUSION

Author-nominated reviewers produced reviews of similar quality to editor-chosen reviewers but were more likely to recommend acceptance during the initial stages of peer review.

摘要

背景

BioMed Central（BMC）要求作者在投稿时推荐四名审稿人。寻找审稿人的编辑会将这些推荐作为一个来源。BMC系列医学期刊的审稿过程是公开的——作者和审稿人彼此知道对方的身份——尽管审稿人可以向编辑提供保密意见。审稿意见会与被接受的文章一起发表，这样读者就能看到审稿人的名字和建议。我们的目的是在一份仅在线出版的医学期刊中，比较作者提名审稿人（ANR）和编辑选定审稿人（ECR）在审稿质量以及关于投稿的建议方面的表现。

方法

对BMC系列医学期刊连续100篇投稿的成对审稿意见（一篇由作者提名审稿人，一篇由编辑选定审稿人，并做出最终决定），由两名评分者进行评估，评分者对审稿人类型不知情，使用经过验证的审稿质量工具（RQI），该工具在5点李克特量表上对7个项目进行评分。评分者在前20对之后讨论他们的评分（保持审稿人类型保密），并解决评分和解释方面的主要差异，以提高评分者间的可靠性。还比较了审稿人的建议。

结果

审稿人来源对审稿质量（ANR的平均RQI得分（±标准差）为2.24±0.55，ECR为2.34±0.54）或语气（附加问题的平均得分ANR为2.72，ECR为2.82）（两种情况的最高分均为5）没有影响。然而，在初步审稿后，作者提名的审稿人比编辑选定的审稿人更有可能推荐接受（47比35），而不太可能推荐拒绝（10比23）（p<0.001）。然而，到最终审稿阶段（即作者对审稿意见做出回应之后），ANR和ECR的建议相似（65比66接受，10比14拒绝，p = 0.47）。在两个审稿阶段，两组中无法决定是否接受的审稿人数量相似。