Bérard A, Andreu N, Tétrault J, Niyonsenga T, Myhal D
Harvard Medical School, Brigham and Women's Hospital, Division of Pharmacoepidemiology and Pharmacoeconomics, Boston, MA, USA.
Ann Epidemiol. 2000 Nov;10(8):498-503. doi: 10.1016/s1047-2797(00)00069-7.
This study estimates the inter-rater and test-retest reliability of Chalmers' quality score scale in the context of bone mass loss and fracture rate in postmenopausal women.
An exhaustive literature search was performed on Medline to locate clinical trials studying the effect of medication use on bone mass loss and fracture rate in postmenopausal women. Twenty articles were randomly selected and four raters independently assessed the quality of each article with Chalmers' scale. Among the 20 articles, 10 were blinded on authors' names, journal, year of publication and source of funding. Raters were also asked to assess all 20 articles one more time, two months after the first evaluation. Intraclass (ICC) and test-retest correlation coefficients were calculated.
The overall inter-rater ICC was 0.66 0.55, 0.79. The overall test-retest reliability of Chalmers' scale was 0.81 0.67, 0. 98. When ratings were stratified according to articles' blinding status, blinded assessments generated a smaller inter-rater ICC than non-blinded assessments: 0.30 0.17, 0.53 vs. 0.80 0. 71, 0.90. In addition, analyzing sub-scales separately generated different estimates of reliability.
This study shows that the reliability of the quality scale developed by Chalmers substantially varies between sub-scales, and is highly dependent on articles' blinding status. The possibility of bias in rating non-blinded articles can not be ruled out. The reliability of the scale can also be dependent on the outcome studied.
本研究在绝经后女性骨量流失和骨折率的背景下，评估查尔默斯质量评分量表的评分者间信度和重测信度。
在Medline上进行了详尽的文献检索，以查找研究药物使用对绝经后女性骨量流失和骨折率影响的临床试验。随机选择20篇文章，由4名评分者使用查尔默斯量表独立评估每篇文章的质量。在这20篇文章中，10篇对作者姓名、期刊、出版年份和资金来源进行了盲法处理。评分者还被要求在首次评估两个月后，再次对所有20篇文章进行评估。计算组内相关系数（ICC）和重测相关系数。
总体评分者间ICC为0.660.55, 0.79。查尔默斯量表的总体重测信度为0.810.67, 0.98。当根据文章的盲法状态进行分层评分时，盲法评估产生的评分者间ICC低于非盲法评估：0.300.17, 0.53对0.800.71, 0.90。此外，分别分析子量表会得出不同的信度估计值。
本研究表明，查尔默斯制定的质量量表的信度在子量表之间存在很大差异，并且高度依赖于文章的盲法状态。不能排除对非盲法文章评分时存在偏差的可能性。该量表的信度也可能取决于所研究的结果。
