Department of Nursing, School of Medicine, Inha University , Incheon, Korea ; Division of General Internal Medicine, Brigham and Women's Hospital , Boston, MA, USA ; Harvard Medical School , Boston, MA, USA.
Division of General Internal Medicine, Brigham and Women's Hospital , Boston, MA, USA ; Harvard Medical School , Boston, MA, USA ; Department of Emergency Medicine, University of Ulsan College of Medicine, Asan Medical Center , Seoul, Korea ; Department of Biomedical Informatics, Asan Medical Center , Seoul, Korea.
Appl Clin Inform. 2014 Jun 18;5(2):571-88. doi: 10.4338/ACI-2014-01-RA-0005. eCollection 2014.
The Instrument for Evaluating Human-Factor Principles in Medication-Related Decision Support Alerts (I-MeDeSA) was developed recently in the US with a view towards improving considerations of human-factor principles when designing alerts for clinical decision support (CDS) systems. This study evaluated the generalizability of this tool, in cooperation with its authors, across cultures by applying it to a Korean system. We also examined opportunities to promote user acceptance of the system.
We developed a Korean version of the I-MeDeSA (K-I-MeDeSA) and used it to evaluate drug-drug interaction alerts in a large academic tertiary hospital in Seoul. We involved four reviewers (A, B, C, and D). Two (A and B) conducted the initial independent scoring, while the other two (C and D) performed a final review and assessed feedback from the initial reviewers. The obtained scores were compared with those from 13 previously reported CDS systems. The feedback was summarized qualitatively.
The translation of the I-MeDeSA had excellent interrater agreement in terms of face validity (scale-level content validity index = 0.95). The system's K-I-MeDeSA score was 10 out of 26, with a good agreement between reviewers (κ = 0.77), which showed a lack of human-factor considerations. The reviewers readily identified two of the nine principles that needed primary improvement: prioritization and text-based information. The reviewers also expressed difficulty judging the following four principles: alarm philosophy, visibility, color, and learnability and confusability.
The K-I-MeDeSA was semantically and operationally equivalent to the original tool. Only minor cultural problems were identified, leading the reviewers to suggest the need for clarification of certain words plus a more detailed description of the tool's rationale and exemplars. Further evaluation is needed to empirically assess whether the implementation of changes in an electronic health record system could improve the adoption of CDS alerts.
最近在美国开发了一种用于评估与药物相关决策支持警报中人为因素原则的工具(I-MeDeSA),旨在改善在设计临床决策支持(CDS)系统的警报时对人为因素原则的考虑。本研究通过将其应用于韩国系统,与作者合作评估了该工具在跨文化背景下的通用性。我们还研究了促进系统用户接受度的机会。
我们开发了 I-MeDeSA 的韩文版本(K-I-MeDeSA),并在首尔的一家大型学术三级医院中使用它来评估药物-药物相互作用警报。我们涉及四名评审员(A、B、C 和 D)。两名(A 和 B)进行了初始独立评分,而另外两名(C 和 D)进行了最终审查并评估了初始评审员的反馈。获得的分数与之前报道的 13 个 CDS 系统的分数进行了比较。反馈意见进行了定性总结。
I-MeDeSA 的翻译在表面有效性方面具有出色的评分者间一致性(量表水平内容有效性指数=0.95)。该系统的 K-I-MeDeSA 得分为 26 分中的 10 分,评审员之间的一致性较好(κ=0.77),这表明缺乏人为因素考虑。评审员很容易确定需要主要改进的九个原则中的两个:优先级和基于文本的信息。评审员还表示在判断以下四个原则时存在困难:报警理念、可见性、颜色和可学习性和混淆性。
K-I-MeDeSA 在语义和操作上与原始工具等效。仅发现了一些较小的文化问题,这促使评审员建议需要澄清某些单词,并更详细地描述工具的原理和示例。需要进一步评估,以实证评估在电子健康记录系统中实施更改是否可以提高 CDS 警报的采用率。