Shimizu Ikuo, Tanaka Kazumi, Mori Jun-Ichirou, Yamauchi Aiga, Kato Sawako, Masuda Yuichi, Nakazawa Yuichi, Kanno Hiroyuki
Medical Education, Chiba University Graduate School of Medicine, Chiba, JPN.
Center for Medical Education and Clinical Training, Shinshu University School of Medicine, Matsumoto, JPN.
Cureus. 2024 Nov 19;16(11):e73969. doi: 10.7759/cureus.73969. eCollection 2024 Nov.
Background The growing emphasis on improving patient safety over the past two decades has received more focus in the undergraduate curricula, and the appropriate assessment of patient safety competencies at graduation is crucial in competency-based medical education. However, there is no valid method for assessing patient safety competencies because current assessment methods in medical education focus less on behavior. The objective structured clinical examination (OSCE) is a method to assess clinical performance and has been implemented by medical schools in Japan for summative assessment at graduation. However, stations with sufficient validity to assess patient safety competencies have not yet been developed. Thus, this study aimed to evaluate, under a contemporary validity framework, an OSCE station for assessing patient safety competencies that students are expected to achieve at graduation from medical schools in Japan. Methods A validity argument was conducted using Messick's validity framework, which includes content, response process, relations to other variables, internal structure, and consequences. First, we applied a modified Delphi study to develop OSCE stations for assessing patient safety competencies based on the national model core curriculum at graduation. The panel survey recruited members who have expertise in clinical education and patient safety. The draft stations simulated various situations associated with patient safety. Final-year medical students then took the OSCE. We analyzed the results of the OSCE, compared the scores with those of the clinical reasoning examination, and evaluated its reliability. Results Out of 30 panelists, 22 (73.3%) fully participated in the Delphi rounds. After two Delphi rounds, we established four stations to assess patient safety competencies. They met the content dimension of the validity framework. The OSCE results showed low correlation with clinical reasoning, suggesting that patient competencies cannot be inferred from clinical reasoning. Each station had satisfactory reliability. The entire process minimized possible assessment bias. Conclusions The OSCE scenario designed through the modified Delphi study met the five criteria of Messick's validity framework. The results show that it is a valid strategy for assessing patient safety competencies at graduation.
背景 在过去二十年中，对提高患者安全的日益重视在本科课程中受到了更多关注，而在基于能力的医学教育中，毕业时对患者安全能力的适当评估至关重要。然而，由于医学教育中的当前评估方法较少关注行为，因此没有有效的方法来评估患者安全能力。客观结构化临床考试（OSCE）是一种评估临床能力的方法，日本的医学院校已将其用于毕业时的总结性评估。然而，尚未开发出具有足够效度来评估患者安全能力的考站。因此，本研究旨在在当代效度框架下，评估一个用于评估日本医学院校学生毕业时应具备的患者安全能力的OSCE考站。
方法 使用梅西克效度框架进行效度论证，该框架包括内容、反应过程、与其他变量的关系、内部结构和后果。首先，我们应用改良的德尔菲研究，根据国家毕业时的示范核心课程开发用于评估患者安全能力的OSCE考站。小组调查招募了临床教育和患者安全方面的专家。拟定的考站模拟了与患者安全相关的各种情况。最后一年的医学生随后参加了OSCE。我们分析了OSCE的结果，将分数与临床推理考试的分数进行比较，并评估了其可靠性。
结果 30名小组成员中，22名（73.3%）充分参与了德尔菲轮次。经过两轮德尔菲研究，我们建立了四个考站来评估患者安全能力。它们符合效度框架的内容维度。OSCE结果显示与临床推理的相关性较低，这表明无法从临床推理中推断出患者能力。每个考站都具有令人满意的可靠性。整个过程将可能的评估偏差降至最低。
结论 通过改良的德尔菲研究设计的OSCE情景符合梅西克效度框架的五个标准。结果表明，它是一种在毕业时评估患者安全能力的有效策略。
