Research Center, Faculty of Rehabilitation Medicine, University of Alberta, Edmonton, Alberta, Canada.
J Eval Clin Pract. 2012 Feb;18(1):12-8. doi: 10.1111/j.1365-2753.2010.01516.x. Epub 2010 Aug 4.
The Cochrane Collaboration is strongly encouraging the use of a newly developed tool, the Cochrane Collaboration Risk of Bias Tool (CCRBT), for all review groups. However, the psychometric properties of this tool to date have yet to be described. Thus, the objective of this study was to add information about psychometric properties of the CCRBT including inter-rater reliability and concurrent validity, in comparison with the Effective Public Health Practice Project Quality Assessment Tool (EPHPP).
Both tools were used to assess the methodological quality of 20 randomized controlled trials included in our systematic review of the effectiveness of knowledge translation interventions to improve the management of cancer pain. Each study assessment was completed independently by two reviewers using each tool. We analysed the inter-rater reliability of each tool's individual domains, as well as final grade assigned to each study.
The EPHPP had fair inter-rater agreement for individual domains and excellent agreement for the final grade. In contrast, the CCRBT had slight inter-rater agreement for individual domains and fair inter-rater agreement for final grade. Of interest, no agreement between the two tools was evident in their final grade assigned to each study. Although both tools were developed to assess 'quality of the evidence', they appear to measure different constructs.
Both tools performed quite differently when evaluating the risk of bias or methodological quality of studies in knowledge translation interventions for cancer pain. The newly introduced CCRBT assigned these studies a higher risk of bias. Its psychometric properties need to be more thoroughly validated, in a range of research fields, to understand fully how to interpret results from its application.
科克伦协作组织强烈鼓励所有审查组使用新开发的工具，即科克伦协作组织偏倚风险工具（CCRBT）。然而，迄今为止，该工具的心理测量学特性尚未得到描述。因此，本研究的目的是提供有关 CCRBT 的心理测量学特性的信息，包括内部评估者间信度和同时效度，并与有效公共卫生实践项目质量评估工具（EPHPP）进行比较。
这两种工具都用于评估我们对知识转化干预措施以改善癌症疼痛管理效果的系统评价中纳入的 20 项随机对照试验的方法学质量。每个研究评估均由两名评估者独立使用每种工具完成。我们分析了每个工具的各个领域的评估者间信度，以及对每个研究的最终评分。
EPHPP 对各个领域的内部评估者间信度为中等，对最终评分的信度为优秀。相比之下，CCRBT 对各个领域的内部评估者间信度为轻微，对最终评分的信度为中等。有趣的是，这两种工具在对每个研究的最终评分上没有显示出一致的意见。尽管这两种工具都是为评估“证据质量”而开发的，但它们似乎测量了不同的结构。
在评估癌症疼痛知识转化干预措施的偏倚风险或方法学质量时，这两种工具的表现截然不同。新引入的 CCRBT 认为这些研究的偏倚风险更高。其心理测量学特性需要在更广泛的研究领域中得到更彻底的验证，以全面了解如何解释其应用结果。
