Werneke Mark W, Hart Dennis L, Deutscher Daniel, Stratford Paul W
CentraState Medical Center, Freehold, NJ, USA.
J Man Manip Ther. 2011 Aug;19(3):172-81. doi: 10.1179/2042618611Y.0000000001.
To estimate inter-rater agreement of physical therapists trained in MDT approach and participating in practice-based evidence (PBE) research to identify 72 physical therapy interventions in video demonstrations on a single model and clinical vignettes. PBE is a well designed observational study and demonstrating clinician observational consistency is an important step in conducting PBE research design.
Two physical therapists volunteered to participate in pilot reliability testing and seven other physical therapists trained in McKenzie Mechanical Diagnosis and Therapy (MDT) methods volunteered for the inter-rater chance-corrected agreement study. All therapists identified interventions presented within 52 videos and 5 written clinical vignettes describing 20 more intervention techniques. Therapists independently identified all interventions. We assessed inter-rater chance-corrected agreement of therapists' ability to identify intervention techniques using Kappa coefficients with associated 95% confidence intervals and indices for bias and prevalence.
Of the 147 kappa coefficients estimated, 7% were ⩽0·6, 10% were >0·6 and ⩽0·8, and 83% were >0·8. Agreement was lowest for identifying cognitive behavioral techniques (median kappa = 0·79). The minimum and maximum prevalence and bias indices were 0·33 and 0·85 and 0 and 0·33, respectively suggesting kappa coefficient estimates were strong. Generalized kappa coefficients ranged from 0·73 to 1·00.
Results provide evidence that substantial to almost perfect inter-rater agreement could be expected when trained therapists identify physical therapy interventions used for patients with spinal impairments from staged videos and vignettes. This may be helpful to reassure clinicians of the quality of the reporting of intervention(s) performed when conducting multivariable analyses in future pragmatic PBE studies. Additional studies are needed to test whether these results can be validated using larger groups of therapists, trained and not trained in MDT methods, as well as examining different methods to examine inter-rater agreement for identifying diverse interventions commonly used for managing patients during routine practice.
评估接受多学科团队（MDT）方法培训并参与基于实践的证据（PBE）研究的物理治疗师之间的评分者间一致性，以识别单一模型视频演示和临床案例中的72种物理治疗干预措施。PBE是一项精心设计的观察性研究，证明临床医生的观察一致性是进行PBE研究设计的重要一步。
两名物理治疗师自愿参与初步可靠性测试，另外七名接受麦肯齐机械诊断与治疗（MDT）方法培训的物理治疗师自愿参与评分者间机会校正一致性研究。所有治疗师识别了52个视频和5个书面临床案例中呈现的干预措施，这些案例描述了另外20种干预技术。治疗师独立识别所有干预措施。我们使用卡方系数及其相关的95%置信区间以及偏差和患病率指数，评估治疗师识别干预技术能力的评分者间机会校正一致性。
在估计的147个卡方系数中，7%≤0.6，10%＞0.6且≤0.8，83%＞0.8。识别认知行为技术的一致性最低（中位数卡方=0.79）。最小和最大患病率及偏差指数分别为0.33和0.85以及0和0.33，表明卡方系数估计值较强。广义卡方系数范围为0.73至1.00。
结果表明，当训练有素的治疗师从分阶段的视频和案例中识别用于脊柱损伤患者的物理治疗干预措施时，可以预期评分者间的一致性达到实质性到几乎完美的程度。这可能有助于让临床医生放心，在未来的实用PBE研究中进行多变量分析时，所报告的干预措施的质量。需要进行更多研究，以测试这些结果是否可以通过更大组的治疗师（无论是否接受MDT方法培训）进行验证，以及研究不同方法来检查评分者间一致性，以识别常规实践中用于管理患者的各种不同干预措施。
