Roy Marguerite, Lockyer Jocelyn, Touchie Claire
Roy: Adjunct Professor, Department of Innovation in Medical Education, University of Ottawa, Ottawa, Canada. Lockyer: Professor, Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Calgary, Canada. Touchie: Professor of Medicine, Department of Medicine, University of Ottawa and the Ottawa Hospital, Ottawa, Canada, and Department of Innovation in Medical Education, Medical Council of Canada, Ottawa, Canada .
J Contin Educ Health Prof. 2023;43(3):155-163. doi: 10.1097/CEH.0000000000000454. Epub 2022 Jul 6.
Evaluation of quality improvement programs shows variable impact on physician performance often neglecting to examine how implementation varies across contexts and mechanisms that affect uptake. Realist evaluation enables the generation, refinement, and testing theories of change by unpacking what works for whom under what circumstances and why. This study used realist methods to explore relationships between outcomes, mechanisms (resources and reasoning), and context factors of a national multisource feedback (MSF) program.
Linked data for 50 physicians were examined to determine relationships between action plan completion status (outcomes), MSF ratings, MSF comments and prescribing data (resource mechanisms), a report summarizing the conversation between a facilitator and physician (reasoning mechanism), and practice risk factors (context). Working backward from outcomes enabled exploration of similarities and differences in mechanisms and context.
The derived model showed that the completion status of plans was influenced by interaction of resource and reasoning mechanisms with context mediating the relationships. Two patterns were emerged. Physicians who implemented all their plans within six months received feedback with consistent messaging, reviewed data ahead of facilitation, coconstructed plan(s) with the facilitator, and had fewer risks to competence (dyscompetence). Physicians who were unable to implement any plans had data with fewer repeated messages and did not incorporate these into plans, had difficult plans, or needed to involve others and were physician-led, and were at higher risk for dyscompetence.
Evaluation of quality improvement initiatives should examine program outcomes taking into consideration the interplay of resources, reasoning, and risk factors for dyscompetence.
对质量改进项目的评估显示，其对医生表现的影响各不相同，往往忽略了考察实施过程在不同背景以及影响接受程度的机制方面是如何变化的。现实主义评估通过剖析在何种情况下、为何对何人有效，从而生成、完善并检验变革理论。本研究采用现实主义方法，探讨了一项全国多源反馈（MSF）项目的结果、机制（资源和推理）与背景因素之间的关系。
对50名医生的关联数据进行了检查，以确定行动计划完成状态（结果）、MSF评分、MSF评论与处方数据（资源机制）、一份总结协调员与医生之间对话的报告（推理机制）以及执业风险因素（背景）之间的关系。从结果出发进行逆向分析，有助于探索机制和背景方面的异同。
得出的模型表明，计划的完成状态受资源和推理机制与起到关系中介作用的背景之间相互作用的影响。出现了两种模式。在六个月内实施了所有计划的医生收到了信息一致的反馈，在协调之前查看了数据，与协调员共同制定了计划，且能力不足（不胜任）的风险较低。无法实施任何计划的医生所收到的数据中重复信息较少，且未将这些信息纳入计划，计划难度较大，或者需要让其他人参与且以医生为主导，他们能力不足的风险较高。
对质量改进举措的评估应在考虑资源、推理以及能力不足风险因素相互作用的情况下考察项目结果。
