适应性弗雷斯诺测试在多个评分者之间的评分者间信度。

Interrater Reliability of the Adapted Fresno Test across Multiple Raters.

作者信息

Lizarondo Lucylynn M, Grimmer Karen, Kumar Saravana

机构信息

International Centre for Allied Health Evidence.

International Centre for Allied Health Evidence ; School of Health Sciences, University of South Australia, Adelaide.

出版信息

Physiother Can. 2013 Spring;65(2):135-40. doi: 10.3138/ptc.2012-15.

DOI:10.3138/ptc.2012-15

PMID:24403674

原文链接:https://pmc.ncbi.nlm.nih.gov/articles/PMC3673791/

Abstract

PURPOSE

The Adapted Fresno Test (AFT) is a seven-item instrument for assessing knowledge and skills in the major domains of evidence-based practice (EBP), including formulating clinical questions and searching for and critically appraising research evidence. This study examined the interrater reliability of the AFT using several raters with different levels of professional experience.

METHOD

The AFT was completed by physiotherapists and occupational therapists, and a random sample of 12 tests was scored by four raters with different levels of professional experience. Interrater reliability was calculated using intra-class correlation coefficients (ICC [2, 1]) for the individual AFT items and the total AFT score.

RESULTS

Interrater reliability was moderate to excellent for items 1 and 7 (ICC=0.63-0.95). Questionable levels of reliability among raters were found for other items and for the total score. For these items, the raters were clustered into two groups-"experienced" and "inexperienced"-and then examined for reliability. The reliability estimates for rater 1 and rater 2 ("inexperienced") increased slightly for items 2 and 5 and for the total score, but not for other items. For raters 3 and 4 ("experienced"), ICCs increased considerably, indicating excellent reliability for all items and for the total score (0.80-0.99), except for item 4, which showed a further decrease in ICC.

CONCLUSION

Use of the AFT to assess knowledge and skills in EBP may be problematic unless raters are carefully selected and trained.

摘要

目的

适应性弗雷斯诺测试（AFT）是一种包含七个项目的工具，用于评估循证实践（EBP）主要领域的知识和技能，包括提出临床问题、检索和批判性评价研究证据。本研究使用了具有不同专业经验水平的多名评分者，检验了AFT的评分者间信度。

方法

物理治疗师和职业治疗师完成了AFT测试，由四名具有不同专业经验水平的评分者对随机抽取的12份测试进行评分。使用组内相关系数（ICC[2,1]）计算AFT各个项目和AFT总分的评分者间信度。

结果

项目1和项目7的评分者间信度为中等至优秀（ICC=0.63-0.95）。其他项目和总分的评分者间信度水平存疑。对于这些项目，评分者被分为“经验丰富”和“经验不足”两组，然后检验信度。评分者1和评分者2（“经验不足”）对项目2和项目5以及总分的信度估计略有提高，但其他项目没有。对于评分者3和评分者4（“经验丰富”），ICC大幅提高，表明除项目4的ICC进一步下降外，所有项目和总分的信度都非常高（0.80-0.99）。