Jones Emma K, Hultman Gretchen, Schmoke Kristine, Ninkovic Ivana, Dodge Sarah, Bahr Matthew, Melton Genevieve B, Marquard Jenna, Tignanelli Christopher J
Department of Surgery, University of Minnesota, Minneapolis, MN.
Institute for Health Informatics, University of Minnesota, Minneapolis, MN.
Surgery. 2022 Nov;172(5):1537-1548. doi: 10.1016/j.surg.2022.05.037. Epub 2022 Aug 26.
Trauma clinical decision support systems improve adherence with evidence-based practice but suffer from poor usability and the lack of a user-centered design. The objective of this study was to compare the effectiveness of user and expert-driven usability testing methods to detect usability issues in a rib fracture clinical decision support system and identify guiding principles for trauma clinical decision support systems.
A user-driven and expert-driven usability investigation was conducted using a clinical decision support system developed for patients with rib fractures. The user-driven usability evaluation was as follows: 10 clinicians were selected for simulation-based usability testing using snowball sampling, and each clinician completed 3 simulations using a video-conferencing platform. End-users participated in a novel team-based approach that simulated realistic clinical workflows. The expert-driven heuristic evaluation was as follows: 2 usability experts conducted a heuristic evaluation of the clinical decision support system using 10 common usability heuristics. Usability issues were identified, cataloged, and ranked for severity using a 4-level ordinal scale. Thematic analysis was utilized to categorize the identified usability issues.
Seventy-nine usability issues were identified; 63% were identified by experts and 48% by end-users. Notably, 58% of severe usability issues were identified by experts alone. Only 11% of issues were identified by both methods. Five themes were identified that could guide the design of clinical decision support systems-transparency, functionality and integration into workflow, automated and noninterruptive, flexibility, and layout and appearance. Themes were preferentially identified by different methods.
We found that a dual-method usability evaluation involving usability experts and end-users drastically improved detection of usability issues over single-method alone. We identified 5 themes to guide trauma clinical decision support system design. Performing usability testing via a remote video-conferencing platform facilitated multi-site involvement despite a global pandemic.
创伤临床决策支持系统可提高循证实践的依从性，但存在可用性差和缺乏以用户为中心设计的问题。本研究的目的是比较用户驱动和专家驱动的可用性测试方法在检测肋骨骨折临床决策支持系统可用性问题方面的有效性，并确定创伤临床决策支持系统的指导原则。
使用为肋骨骨折患者开发的临床决策支持系统进行了用户驱动和专家驱动的可用性调查。用户驱动的可用性评估如下：通过滚雪球抽样选择10名临床医生进行基于模拟的可用性测试，每位临床医生使用视频会议平台完成3次模拟。终端用户采用一种新颖的基于团队的方法，模拟现实临床工作流程。专家驱动的启发式评估如下：2名可用性专家使用10条常见的可用性启发式规则对临床决策支持系统进行启发式评估。识别可用性问题，进行分类，并使用4级有序量表对严重程度进行排名。采用主题分析对识别出的可用性问题进行分类。
共识别出79个可用性问题；63%由专家识别，48%由终端用户识别。值得注意的是，58%的严重可用性问题仅由专家识别。两种方法都识别出的问题仅占11%。确定了五个可指导临床决策支持系统设计的主题——透明度、功能以及融入工作流程、自动化和非干扰性、灵活性以及布局和外观。不同方法优先识别出不同主题。
我们发现，与单独使用单一方法相比，由可用性专家和终端用户参与的双方法可用性评估极大地提高了可用性问题的检测率。我们确定了5个主题以指导创伤临床决策支持系统的设计。尽管处于全球大流行期间，但通过远程视频会议平台进行可用性测试促进了多地点参与。
