Social Psychology and Research on Higher Education, ETH Zurich, Zurich, Switzerland.
PLoS One. 2010 Jun 28;5(6):e11344. doi: 10.1371/journal.pone.0011344.
High predictive validity--that is, a strong association between the outcome of peer review (usually, reviewers' ratings) and the scientific quality of a manuscript submitted to a journal (measured as citations of the later published paper)--does not as a rule suffice to demonstrate the usefulness of peer review for the selection of manuscripts. To assess usefulness, it is important to include in addition the base rate (proportion of submissions that are fundamentally suitable for publication) and the selection rate (the proportion of submissions accepted).
METHODOLOGY/PRINCIPAL FINDINGS: Taking the example of the high-impact journal Angewandte Chemie International Edition (AC-IE), we present a general approach for determining the usefulness of peer reviews for the selection of manuscripts for publication. The results of our study show that peer review is useful: 78% of the submissions accepted by AC-IE are correctly accepted for publication when the editor's decision is based on one review, 69% of the submissions are correctly accepted for publication when the editor's decision is based on two reviews, and 65% of the submissions are correctly accepted for publication when the editor's decision is based on three reviews.
CONCLUSIONS/SIGNIFICANCE: The paper points out through what changes in the selection rate, base rate or validity coefficient a higher success rate (utility) in the AC-IE selection process could be achieved.
高预测效度——即同行评审结果（通常为评审员的评分）与提交给期刊的手稿的科学质量之间的强关联（以后续发表论文的引用次数衡量）——通常不足以证明同行评审对于选择手稿的有用性。为了评估有用性，除了基本接受率（适合出版的提交比例）和选择率（接受的提交比例）之外，还需要包括在内。
方法/主要发现：以高影响力期刊《德国应用化学》（Angewandte Chemie International Edition，AC-IE）为例，我们提出了一种确定同行评审对于选择出版手稿的有用性的一般方法。我们的研究结果表明，同行评审是有用的：当编辑的决策基于一次评审时，AC-IE 接受的 78%的投稿被正确接受出版，当编辑的决策基于两次评审时，69%的投稿被正确接受出版，当编辑的决策基于三次评审时，65%的投稿被正确接受出版。
结论/意义：本文指出，通过选择率、基础率或有效性系数的变化，可以提高 AC-IE 选择过程的更高成功率（效用）。
