Schmettow Martin, Schnittker Raphaela, Schraagen Jan Maarten
Department of Cognitive Psychology and Ergonomics, University of Twente, Enschede, The Netherlands.
Monash University Accident Research Centre, Monash University, Clayton, Australia.
J Biomed Inform. 2017 May;69:99-114. doi: 10.1016/j.jbi.2017.03.010. Epub 2017 Mar 21.
This paper proposes and demonstrates an extended protocol for usability validation testing of medical devices. A review of currently used methods for the usability evaluation of medical devices revealed two main shortcomings. Firstly, the lack of methods to closely trace the interaction sequences and derive performance measures. Secondly, a prevailing focus on cross-sectional validation studies, ignoring the issues of learnability and training. The U.S. Federal Drug and Food Administration's recent proposal for a validation testing protocol for medical devices is then extended to address these shortcomings: (1) a novel process measure 'normative path deviations' is introduced that is useful for both quantitative and qualitative usability studies and (2) a longitudinal, completely within-subject study design is presented that assesses learnability, training effects and allows analysis of diversity of users. A reference regression model is introduced to analyze data from this and similar studies, drawing upon generalized linear mixed-effects models and a Bayesian estimation approach. The extended protocol is implemented and demonstrated in a study comparing a novel syringe infusion pump prototype to an existing design with a sample of 25 healthcare professionals. Strong performance differences between designs were observed with a variety of usability measures, as well as varying training-on-the-job effects. We discuss our findings with regard to validation testing guidelines, reflect on the extensions and discuss the perspectives they add to the validation process.
本文提出并演示了一种用于医疗设备可用性验证测试的扩展协议。对当前用于医疗设备可用性评估的方法进行回顾后发现了两个主要缺点。首先，缺乏密切跟踪交互序列并得出性能指标的方法。其次，普遍侧重于横断面验证研究，而忽略了可学习性和培训问题。然后对美国食品药品监督管理局最近提出的医疗设备验证测试协议进行了扩展，以解决这些缺点：（1）引入了一种新颖的过程指标“规范路径偏差”，该指标对定量和定性可用性研究均有用；（2）提出了一种纵向的、完全在受试者内部的研究设计，该设计可评估可学习性、培训效果并允许分析用户的多样性。引入了一个参考回归模型，以利用广义线性混合效应模型和贝叶斯估计方法分析来自该研究及类似研究的数据。在一项将新型注射器输液泵原型与现有设计进行比较的研究中，对25名医疗保健专业人员进行了抽样，实施并演示了扩展协议。在各种可用性指标以及不同的在职培训效果方面，观察到了设计之间的显著性能差异。我们讨论了关于验证测试指南的研究结果，反思了这些扩展，并讨论了它们为验证过程带来的前景。
