Department of Statistics and Operations Research, Barcelona-Tech, UPC, c/ Jordi Girona 1-3, 08034, Barcelona, Spain.
INSERM, U1153 Epidemiology and Biostatistics Sorbonne Paris Cité Research Center (CRESS), Methods of therapeutic evaluation of chronic diseases Team (METHODS), F-75014, Paris, France.
BMC Med Res Methodol. 2019 Mar 6;19(1):48. doi: 10.1186/s12874-019-0688-x.
A strong need exists for a validated tool that clearly defines peer review report quality in biomedical research, as it will allow evaluating interventions aimed at improving the peer review process in well-performed trials. We aim to identify and describe existing tools for assessing the quality of peer review reports in biomedical research.
We conducted a methodological systematic review by searching PubMed, EMBASE (via Ovid) and The Cochrane Methodology Register (via The Cochrane Library) as well as Google® for all reports in English describing a tool for assessing the quality of a peer review report in biomedical research. Data extraction was performed in duplicate using a standardized data extraction form. We extracted information on the structure, development and validation of each tool. We also identified quality components across tools using a systematic multi-step approach and we investigated quality domain similarities among tools by performing hierarchical, complete-linkage clustering analysis.
We identified a total number of 24 tools: 23 scales and 1 checklist. Six tools consisted of a single item and 18 had several items ranging from 4 to 26. None of the tools reported a definition of 'quality'. Only 1 tool described the scale development and 10 provided measures of validity and reliability. Five tools were used as an outcome in a randomized controlled trial (RCT). Moreover, we classified the quality components of the 18 tools with more than one item into 9 main quality domains and 11 subdomains. The tools contained from two to seven quality domains. Some domains and subdomains were considered in most tools such as the detailed/thorough (11/18) nature of reviewer's comments. Others were rarely considered, such as whether or not the reviewer made comments on the statistical methods (1/18).
Several tools are available to assess the quality of peer review reports; however, the development and validation process is questionable and the concepts evaluated by these tools vary widely. The results from this study and from further investigations will inform the development of a new tool for assessing the quality of peer review reports in biomedical research.
在生物医学研究中，需要一个经过验证的工具来明确界定同行评审报告的质量，因为这将有助于评估旨在改善表现良好的试验中同行评审过程的干预措施。我们旨在确定和描述用于评估生物医学研究中同行评审报告质量的现有工具。
我们通过搜索 PubMed、EMBASE（通过 Ovid）和 The Cochrane Methodology Register（通过 The Cochrane Library）以及 Google®，查找所有以英文描述用于评估生物医学研究中同行评审报告质量的工具的报告，进行了一项方法学系统评价。使用标准化的数据提取表格进行了重复的数据提取。我们提取了每个工具的结构、开发和验证信息。我们还使用系统的多步骤方法确定了工具之间的质量组成部分，并通过执行分层、完全链接聚类分析来研究工具之间的质量域相似性。
我们共确定了 24 种工具：23 个量表和 1 个清单。6 种工具只有一个项目，18 种工具有 4 到 26 个项目。没有一种工具报告了“质量”的定义。只有 1 种工具描述了量表的开发，10 种工具提供了有效性和可靠性的度量。有 5 种工具被用作随机对照试验（RCT）的结果。此外，我们将 18 个以上项目的工具的质量组成部分分为 9 个主要质量域和 11 个子域。这些工具包含 2 到 7 个质量域。大多数工具都考虑了一些域和子域，例如评审员评论的详细/全面（11/18）性质。其他很少被考虑，例如评审员是否对统计方法发表了评论（1/18）。
有几种工具可用于评估同行评审报告的质量；然而，开发和验证过程存在疑问，这些工具评估的概念差异很大。这项研究和进一步调查的结果将为开发一种新的工具提供信息，用于评估生物医学研究中同行评审报告的质量。
