Couturier Jennifer, Kimber Melissa, Barwick Melanie, McVey Gail, Findlay Sheri, Webb Cheryl, Niccols Alison, Lock James
Department of Psychiatry & Behavioural Neurosciences, McMaster University, Hamilton, Canada.
Offord Centre for Child Studies, McMaster University, Hamilton, Canada.
J Eat Disord. 2021 Jan 14;9(1):12. doi: 10.1186/s40337-020-00366-5.
Fidelity is an essential component for evaluating the clinical and implementation outcomes related to delivery of evidence-based practices (EBPs). Effective measurement of fidelity requires clinical buy-in, and as such, requires a process that is not burdensome for clinicians and managers. As part of a larger implementation study, we examined fidelity to Family-Based Treatment (FBT) measured by several different raters including an expert, a peer, therapists themselves, and parents, with a goal of determining a pragmatic, reliable and efficient method to capture treatment fidelity to FBT.
Each therapist audio-recorded at least one FBT case and submitted recordings from session 1, 2, and 3 from phase 1, plus one additional session from phase 1, two sessions from phase 2, and one session from phase 3. These submitted files were rated by an expert and a peer rater using a validated FBT fidelity measure. As well, therapists and parents rated fidelity immediately following each session and submitted ratings to the research team. Inter-observer reliability was calculated for each item using the intraclass correlation coefficient (ICC), comparing the expert ratings to ratings from each of the other raters (parents, therapists, and peer). Mean scale scores were compared using repeated measures ANOVA.
Intraclass correlation coefficients revealed that agreement was the best between expert and peer, with excellent, good, or fair agreement in 7 of 13 items from session 1, 2 and 3. There were only four such values when comparing expert to parent agreement, and two such values comparing expert to therapist ratings. The rest of the ICC values indicated poor agreement. Scale level analysis indicated that expert fidelity ratings for phase 1 treatment sessions scores were significantly higher than the peer ratings and, that parent fidelity ratings tended to be significantly higher than the other raters across all three treatment phases. There were no significant differences between expert and therapist mean scores.
There may be challenges inherent in parents rating fidelity accurately. Peer rating or therapist self-rating may be considered pragmatic, efficient, and reliable approaches to fidelity assessment for real-world clinical settings.
保真度是评估与循证实践（EBP）实施相关的临床和实施结果的重要组成部分。对保真度进行有效测量需要临床人员的认可，因此，需要一个对临床医生和管理人员来说负担不重的过程。作为一项更大规模实施研究的一部分，我们研究了由包括专家、同行、治疗师本人和家长在内的不同评分者对基于家庭治疗（FBT）的保真度，目的是确定一种实用、可靠且高效的方法来获取对FBT治疗保真度的测量。
每位治疗师至少录制了一个FBT案例，并提交了来自第1阶段第1、2和3节的录音，以及第1阶段的另外一节、第2阶段的两节和第3阶段的一节录音。这些提交的文件由一名专家和一名同行评分者使用经过验证的FBT保真度测量方法进行评分。此外，治疗师和家长在每次治疗结束后立即对保真度进行评分，并将评分提交给研究团队。使用组内相关系数（ICC）计算每个项目的观察者间信度，将专家评分与其他评分者（家长、治疗师和同行）的评分进行比较。使用重复测量方差分析比较平均量表得分。
组内相关系数显示，专家和同行之间的一致性最好，在第1、2和3节的13个项目中有7个项目的一致性为优秀、良好或中等。将专家与家长的一致性进行比较时，只有4个这样的值，将专家与治疗师的评分进行比较时，有2个这样的值。其余的ICC值表明一致性较差。量表水平分析表明，第1阶段治疗节次的专家保真度评分显著高于同行评分；在所有三个治疗阶段，家长的保真度评分往往显著高于其他评分者。专家和治疗师的平均得分之间没有显著差异。
家长准确评定保真度可能存在内在挑战。对于现实世界的临床环境，同行评分或治疗师自评可能被认为是实用、高效且可靠的保真度评估方法。
