Scientific Peer Advisory and Review Services, American Institute of Biological Sciences, Herndon, VA, USA.
Washington State University, Psychology, Vancouver, WA, USA.
Sci Eng Ethics. 2021 Mar 17;27(2):18. doi: 10.1007/s11948-021-00295-9.
The primary goal of the peer review of research grant proposals is to evaluate their quality for the funding agency. An important secondary goal is to provide constructive feedback to applicants for their resubmissions. However, little is known about whether review feedback achieves this goal. In this paper, we present a multi-methods analysis of responses from grant applicants regarding their perceptions of the effectiveness and appropriateness of peer review feedback they received from grant submissions. Overall, 56-60% of applicants determined the feedback to be appropriate (fair, well-written, and well-informed), although their judgments were more favorable if their recent application was funded. Importantly, independent of funding success, women found the feedback better written than men, and more white applicants found the feedback to be fair than non-white applicants. Also, perceptions of a variety of biases were specifically reported in respondents' feedback. Less than 40% of applicants found the feedback to be very useful in informing their research and improving grantsmanship and future submissions. Further, negative perceptions of the appropriateness of review feedback were positively correlated with more negative perceptions of feedback usefulness. Importantly, respondents suggested that highly competitive funding pay-lines and poor inter-panel reliability limited the usefulness of review feedback. Overall, these results suggest that more effort is needed to ensure that appropriate and useful feedback is provided to all applicants, bolstering the equity of the review process and likely improving the quality of resubmitted proposals.
研究资助提案同行评审的主要目标是评估其对资助机构的质量。一个重要的次要目标是为申请人的重新提交提供建设性的反馈。然而，对于评审反馈是否达到这一目标，人们知之甚少。在本文中，我们提出了一种多方法分析，分析了资助申请人对他们从资助申请中收到的同行评审反馈的有效性和适当性的看法。总体而言，56-60%的申请人认为反馈是适当的（公平、书写良好且消息灵通），尽管如果他们最近的申请获得了资助，他们的判断更为有利。重要的是，独立于资助成功与否，女性认为反馈的书写比男性更好，而更多的白人申请人认为反馈是公平的，而非白人申请人则认为反馈是公平的。此外，在受访者的反馈中还特别报告了对各种偏见的看法。不到 40%的申请人认为反馈在告知他们的研究、提高资助能力和未来提交方面非常有用。此外，对评审反馈适当性的负面看法与对反馈有用性的负面看法呈正相关。重要的是，受访者认为，竞争激烈的资助支付线和较差的小组间可靠性限制了评审反馈的有用性。总体而言，这些结果表明，需要付出更多努力，以确保向所有申请人提供适当和有用的反馈，从而增强评审过程的公平性，并可能提高重新提交提案的质量。
