Glissmeyer Eric W, Ziniel Sonja I, Moses James
J Grad Med Educ. 2014 Jun;6(2):284-91. doi: 10.4300/JGME-D-13-00221.1.
Assessing the effectiveness of quality improvement curricula is important to improving this area of resident education.
To assess the ability of the Quality Improvement Knowledge Application Tool (QIKAT) to differentiate between residents who were provided instruction in QI and those who were not, when scored by individuals not involved in designing the QIKAT, its scoring rubric, or QI curriculum instruction.
The QIKAT and a 9-item self-assessment of QI proficiency were administered to an intervention and a control group. The intervention was a longitudinal curriculum consisting of 8 hours of didactic QI training and 6 workshops providing just-in-time training for resident QI projects. Two uninvolved faculty scored the QIKAT.
A total of 33 residents in the intervention group and 27 in the control group completed the baseline and postcurriculum QIKAT and self-assessment. QIKAT mean intervention group scores were significantly higher than mean control group scores postcurriculum (P < .001). Absolute QIKAT differences were small (of 15 points, intervention group improved from a mean score of 12.8 to 13.2). Interrater agreement as measured by kappa test was low (0.09). Baseline self-assessment showed no differences, and after instruction, the intervention group felt more proficient in QI knowledge than controls in 4 of 9 domains tested.
The QIKAT detected a statistically significant improvement postintervention, but the absolute differences were small. Self-reported gain in QI knowledge and proficiency agreed with the results of the QIKAT. However, QIKAT limitations include poor interrater agreement and a scoring rubric that lacks specificity. Programs considering using QIKAT to assess curricula should understand these limitations.
评估质量改进课程的有效性对于改善住院医师教育这一领域至关重要。
当由未参与设计质量改进知识应用工具（QIKAT）、其评分标准或质量改进课程教学的人员进行评分时，评估QIKAT区分接受过质量改进教学的住院医师和未接受过此类教学的住院医师的能力。
对一个干预组和一个对照组实施QIKAT以及一项9项的质量改进能力自我评估。干预措施是一门纵向课程，包括8小时的质量改进理论教学培训以及6次为住院医师质量改进项目提供即时培训的工作坊。两名未参与的教员对QIKAT进行评分。
干预组共有33名住院医师，对照组有27名住院医师完成了基线和课程后的QIKAT及自我评估。干预组QIKAT的课程后平均得分显著高于对照组（P < 0.001）。QIKAT的绝对差异较小（满分15分，干预组平均得分从12.8分提高到13.2分）。通过kappa检验测量的评分者间一致性较低（0.09）。基线自我评估无差异，教学后，干预组在9个测试领域中的4个领域中，感觉自己在质量改进知识方面比对照组更精通。
QIKAT检测到干预后有统计学意义的改善，但绝对差异较小。自我报告的质量改进知识和能力的提高与QIKAT的结果一致。然而，QIKAT的局限性包括评分者间一致性差以及评分标准缺乏特异性。考虑使用QIKAT评估课程的项目应了解这些局限性。
