Department of Pharmacology, Inje University College of Medicine, Busan, Korea.
Department of Internal Medicine, Inje University College of Medicine, Busan, Korea.
J Educ Eval Health Prof. 2024;21:27. doi: 10.3352/jeehp.2024.21.27. Epub 2024 Oct 2.
The Dr. LEE Jong-wook Fellowship Program, established by the Korea Foundation for International Healthcare (KOFIH), aims to strengthen healthcare capacity in partner countries. The aim of the study was to develop new performance evaluation indicators for the program to better assess long-term educational impact across various courses and professional roles.
A 3-stage process was employed. First, a literature review of established evaluation models (Kirkpatrick’s 4 levels, context/input/process/product evaluation model, Organization for Economic Cooperation and Development Assistance Committee criteria) was conducted to devise evaluation criteria. Second, these criteria were validated via a 2-round Delphi survey with 18 experts in training projects from May 2021 to June 2021. Third, the relative importance of the evaluation criteria was determined using the analytic hierarchy process (AHP), calculating weights and ensuring consistency through the consistency index and consistency ratio (CR), with CR values below 0.1 indicating acceptable consistency.
The literature review led to a combined evaluation model, resulting in 4 evaluation areas, 20 items, and 92 indicators. The Delphi surveys confirmed the validity of these indicators, with content validity ratio values exceeding 0.444. The AHP analysis assigned weights to each indicator, and CR values below 0.1 indicated consistency. The final set of evaluation indicators was confirmed through a workshop with KOFIH and adopted as the new evaluation tool.
The developed evaluation framework provides a comprehensive tool for assessing the long-term outcomes of the Dr. LEE Jong-wook Fellowship Program. It enhances evaluation capabilities and supports improvements in the training program’s effectiveness and international healthcare collaboration.
由韩国国际保健协力财团（KOFIH）设立的李钟郁博士研究员奖学金项目旨在加强伙伴国家的医疗保健能力。本研究旨在为该项目开发新的绩效评估指标，以更好地评估各种课程和专业角色的长期教育影响。
采用了三阶段的方法。首先，对已建立的评估模型（柯克帕特里克的四级、背景/投入/过程/产品评估模型、经济合作与发展组织援助委员会标准）进行了文献回顾，以制定评估标准。其次，通过 2021 年 5 月至 2021 年 6 月对培训项目的 18 名专家进行了两轮德尔菲调查，对这些标准进行了验证。第三，使用层次分析法（AHP）确定评估标准的相对重要性，通过一致性指数和一致性比率（CR）计算权重，并确保一致性，CR 值低于 0.1 表示可接受的一致性。
文献回顾导致了一个综合的评估模型，产生了 4 个评估领域、20 个项目和 92 个指标。德尔菲调查证实了这些指标的有效性，内容有效性比率值超过 0.444。层次分析法分析为每个指标分配权重，CR 值低于 0.1 表示一致性。通过与 KOFIH 的研讨会最终确定了评估指标，并将其作为新的评估工具采用。
所开发的评估框架为评估李钟郁博士研究员奖学金项目的长期成果提供了一个全面的工具。它提高了评估能力，并支持培训项目的有效性和国际医疗保健合作的改进。
