Wojewodka G, Hurley S, Taylor S J C, Noble A J, Ridsdale L, Goldstein L H
King's College London, Institute of Psychiatry, Psychology and Neuroscience, De Crespigny Park, London, SE5 8AF, UK.
Centre for Primary Care and Public Health, Bart's and the London School of Medicine and Dentistry, Queen Mary University of London, London, UK.
BMC Med Res Methodol. 2017 Jul 11;17(1):100. doi: 10.1186/s12874-017-0373-x.
Complex interventions such as self-management courses are difficult to evaluate due to the many interacting components. The way complex interventions are delivered can influence the effect they have for patients, and can impact the interpretation of outcomes of clinical trials. Implementation fidelity evaluates whether complex interventions are delivered according to protocol. Such assessments have been used for one-to-one psychological interventions; however, the science is still developing for group interventions.
We developed and tested an instrument to measure implementation fidelity of a two-day self-management course for people with epilepsy, SMILE(UK). Using audio recordings, we looked at adherence and competence of course facilitators. Adherence was assessed by checklists. Competence was measured by scoring group interaction, an overall impression score and facilitator "didacticism". To measure "didacticism", we developed a novel way to calculate facilitator speech using computer software. Using this new instrument, implementation fidelity of SMILE(UK) was assessed on three modules of the course, for 28% of all courses delivered.
Using the instrument for adherence, scores from two independent raters showed substantial agreement with weighted Kappa of 0.67 and high percent agreement of 81.2%. For didacticism, the results from both raters were highly correlated with an intraclass coefficient of 0.97 (p < 0.0001). We found that the courses were delivered with a good level of adherence (> 50% of scored items received the maximum of 2 points) and high competence. Groups were interactive (mean score: 1.9-2.0 out of 2) and the overall impression was on average assessed as "good". Didacticism varied from 42% to 93% of total module time and was not associated with the other competence scores.
The instrument devised to measure implementation fidelity was reproducible and easy to use. The courses for the SMILE(UK) study were delivered with a good level of adherence to protocol while not compromising facilitator competence.
ISRCTN57937389 .
自我管理课程等复杂干预措施由于包含诸多相互作用的组成部分，因而难以评估。复杂干预措施的实施方式会影响其对患者产生的效果，且会对临床试验结果的解读产生影响。实施保真度评估复杂干预措施是否按照方案进行实施。此类评估已用于一对一心理干预；然而，针对团体干预的相关科学仍在发展之中。
我们开发并测试了一种工具，用于衡量针对癫痫患者的为期两天的自我管理课程“SMILE（英国）”的实施保真度。通过音频记录，我们考察了课程 facilitator 的依从性和能力。依从性通过清单进行评估。能力通过对小组互动、总体印象评分以及 facilitator 的“说教性”进行评分来衡量。为了衡量“说教性”，我们开发了一种使用计算机软件计算 facilitator 讲话内容的新方法。使用这一新工具，对“SMILE（英国）”课程的三个模块进行了实施保真度评估，涵盖了所交付全部课程的 28%。
使用该工具评估依从性时，两名独立评分者的评分显示出高度一致性，加权 Kappa 系数为 0.67，一致性百分比高达 81.2%。对于说教性，两名评分者的结果高度相关，组内相关系数为 0.97（p < 0.0001）。我们发现课程实施具有较高的依从性（超过 50%的评分项目获得了满分 2 分）且能力水平较高。小组互动性良好（平均得分：满分 2 分中的 1.9 - 2.0 分），总体印象平均被评为“良好”。说教性占总模块时间的比例从 42%到 93%不等，且与其他能力得分无关。
所设计的用于衡量实施保真度的工具具有可重复性且易于使用。“SMILE（英国）”研究中的课程在遵循方案方面具有较高的依从性，同时并未损害 facilitator 的能力。
ISRCTN57937389 。
