Department of Psychobiology and Methodology of Health Sciences, Psychology Faculty, Universitat Autònoma de Barcelona, Carrer de la Fortuna, Edifici B, Bellaterra, Barcelona 08193, Spain.
Department of Business Management, School of Industrial Engineering, Universitat Politècnica de València, Dept. Organización de Empresas, Edificio 7D, Camino de Vera s/n, Valencia 46022, Spain.
J Clin Epidemiol. 2018 Sep;101:61-72. doi: 10.1016/j.jclinepi.2018.05.021. Epub 2018 Jun 2.
The aim of this study was to assess the agreement and compare the performance of three different instruments in assessing risk of bias (RoB) of comparative cohort studies included in a health psychology meta-analysis.
Three tools were applied to 28 primary studies included in the selected meta-analysis: the Newcastle-Ottawa Scale, quality of cohort studies (Q-Coh), and risk of bias in nonrandomized studies of interventions (ROBINS-I).
Interrater agreement varied greatly from tool to tool. For overall RoB, 75% of the studies were rated as low RoB with the Newcastle-Ottawa Scale, 11% of the studies with Q-Coh, and no study was found to be at low RoB using ROBINS-I. No influence of quality ratings on the meta-analysis results was found for any of the tools.
Assessing RoB using the three tools may lead to opposite conclusions, especially at low and high levels of RoB. Domain-based tools (Q-Coh and ROBINS-I) provide a more comprehensive framework for identifying potential sources of bias, which is essential to improving the quality of future research. Both further guidance on the application of RoB tools and improvements in the reporting of primary studies are necessary.
本研究旨在评估三种不同工具在评估纳入健康心理学荟萃分析的比较队列研究偏倚风险（RoB）中的一致性，并比较其性能。
将三种工具应用于选定荟萃分析中包含的 28 项主要研究：纽卡斯尔-渥太华量表（Newcastle-Ottawa Scale）、队列研究质量（Q-Coh）和干预非随机研究偏倚风险（ROBINS-I）。
不同工具的观察者间一致性差异很大。对于整体 RoB，75%的研究被纽卡斯尔-渥太华量表评为低 RoB，11%的研究被 Q-Coh 评为低 RoB，而 ROBINS-I 则没有研究被评为低 RoB。对于任何工具，质量评分均未对荟萃分析结果产生影响。
使用三种工具评估 RoB 可能会得出相反的结论，尤其是在低 RoB 和高 RoB 水平下。基于领域的工具（Q-Coh 和 ROBINS-I）为识别潜在的偏倚来源提供了更全面的框架，这对于提高未来研究的质量至关重要。需要进一步指导 RoB 工具的应用，并改进主要研究的报告。
