Khajouei Reza, Zahiri Esfahani Misagh, Jahani Yunes
Medical Informatics Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran.
Department of Health Information Management and Technology, Faculty of Management and Medical Information Sciences, Kerman University of Medical Sciences, Kerman, Iran.
J Am Med Inform Assoc. 2017 Apr 1;24(e1):e55-e60. doi: 10.1093/jamia/ocw100.
There are several user-based and expert-based usability evaluation methods that may perform differently according to the context in which they are used. The objective of this study was to compare 2 expert-based methods, heuristic evaluation (HE) and cognitive walkthrough (CW), for evaluating usability of health care information systems.
Five evaluators independently evaluated a medical office management system using HE and CW. We compared the 2 methods in terms of the number of identified usability problems, their severity, and the coverage of each method.
In total, 156 problems were identified using the 2 methods. HE identified a significantly higher number of problems related to the "satisfaction" attribute ( P = .002). The number of problems identified using CW concerning the "learnability" attribute was significantly higher than those identified using HE ( P = .005). There was no significant difference between the number of problems identified by HE, based on different usability attributes ( P = .232). Results of CW showed a significant difference between the number of problems related to usability attributes ( P < .0001). The average severity of problems identified using CW was significantly higher than that of HE ( P < .0001).
This study showed that HE and CW do not differ significantly in terms of the number of usability problems identified, but they differ based on the severity of problems and the coverage of some usability attributes. The results suggest that CW would be the preferred method for evaluating systems intended for novice users and HE for users who have experience with similar systems. However, more studies are needed to support this finding.
有几种基于用户和基于专家的可用性评估方法，它们在使用的背景下可能表现不同。本研究的目的是比较两种基于专家的方法，即启发式评估（HE）和认知走查（CW），用于评估医疗保健信息系统的可用性。
五名评估人员使用HE和CW独立评估一个医疗办公室管理系统。我们在识别出的可用性问题数量、其严重程度以及每种方法的覆盖范围方面比较了这两种方法。
使用这两种方法总共识别出156个问题。HE识别出与“满意度”属性相关的问题数量显著更多（P = 0.002）。使用CW识别出的与“可学习性”属性相关的问题数量显著高于使用HE识别出的问题数量（P = 0.005）。基于不同可用性属性，HE识别出的问题数量之间没有显著差异（P = 0.232）。CW的结果显示与可用性属性相关的问题数量之间存在显著差异（P < 0.0001）。使用CW识别出的问题的平均严重程度显著高于HE（P < 0.0001）。
本研究表明，HE和CW在识别出的可用性问题数量方面没有显著差异，但在问题的严重程度和一些可用性属性的覆盖范围方面存在差异。结果表明，CW将是评估面向新手用户的系统的首选方法，而HE适用于有类似系统使用经验的用户。然而，需要更多研究来支持这一发现。
