Miller Erin, Brooks Dina, O'Brien Kelly K, Beavers Lindsay, Stratford Paul, Nonoyama Mika, Mori Brenda
Rehabilitation Sciences Institute, University of Toronto, Toronto, Ontario, Canada.
Department of Physical Therapy, University of Toronto, Toronto, Ontario, Canada.
Physiother Res Int. 2022 Apr;27(2):e1944. doi: 10.1002/pri.1944. Epub 2022 Feb 17.
The Physical Therapy Competence Assessment for Airway Suctioning (PT-CAAS) is a recently developed measure to assess the clinical competence of physiotherapists who perform airway suctioning with adults. The purpose of this study was to assess the inter-rater and intra-rater reliability of the PT-CAAS.
Scoring rules were developed through expert consultation. Reliability was then assessed using nine videos of suctioning performed in a simulated learning environment. A repeated measures design was used, with two replicate sets of measurements made by each participant for all videos. Data were analyzed using a repeated measures model for the concurrent assessment of inter-rater and intra-rater reliability. Participants were physiotherapists with suctioning experience.
Twenty physiotherapists completed initial scoring and re-scoring for all nine videos; their data were included in the analysis. Intraclass correlation coefficients (ICCs) for inter-rater reliability ranged from 0.569 [lower one-sided 95% confidence interval (CI): 0.395; standard error of measurement (SEM): 0.963] for infection control to 0.759 (lower one-sided 95% CI: 0.612; SEM: 0.722) for post-suctioning assessment and care. The inter-rater ICC for overall performance was 0.752 (lower one-sided 95% CI: 0.602; SEM: 0.660). ICCs for intra-rater reliability ranged from 0.759 (lower one-sided 95% CI: 0.197; SEM 0.721) for infection control to 0.860 (lower one-sided 95% CI: 0.544; SEM: 0.550) for post-suctioning assessment and care. The intra-rater ICC for overall performance was 0.867 (lower one-sided 95% CI: 0.559; SEM: 0.483).
Evidence of moderate to good inter-rater and good intra-rater reliability was found; however, the results should be interpreted with caution given the wide CIs and relatively large SEMs. Improved assessor training and assessments of reliability using a larger sample size are recommended.
气道吸痰物理治疗能力评估（PT-CAAS）是最近开发的一种用于评估对成人进行气道吸痰的物理治疗师临床能力的方法。本研究的目的是评估PT-CAAS的评分者间信度和评分者内信度。
通过专家咨询制定评分规则。然后使用在模拟学习环境中进行吸痰的九个视频评估信度。采用重复测量设计，每位参与者对所有视频进行两组重复测量。使用重复测量模型对评分者间信度和评分者内信度进行同步评估。参与者为有吸痰经验的物理治疗师。
20名物理治疗师完成了对所有九个视频的初始评分和重新评分；他们的数据被纳入分析。评分者间信度的组内相关系数（ICC）范围从感染控制方面的0.569[单侧下限95%置信区间（CI）：0.395；测量标准误差（SEM）：0.963]到吸痰后评估与护理方面的0.759（单侧下限95%CI：0.612；SEM：0.722）。整体表现的评分者间ICC为0.752（单侧下限95%CI：0.602；SEM：0.660）。评分者内信度的ICC范围从感染控制方面的0.759（单侧下限95%CI：0.197；SEM 0.721）到吸痰后评估与护理方面的0.860（单侧下限95%CI：0.544；SEM：0.550）。整体表现的评分者内ICC为0.867（单侧下限95%CI：0.559；SEM：0.483）。
发现了评分者间信度为中等至良好以及评分者内信度为良好的证据；然而，鉴于较宽的置信区间和相对较大的测量标准误差，对结果的解释应谨慎。建议改进评估者培训并使用更大样本量进行信度评估。
